Data collection

Gathering information for analysis

This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.
Find sources: "Data collection" – news · newspapers · books · scholar · JSTOR (April 2017) (Learn how and when to remove this message)

|

This article is in list format but may read better as prose. You can help by converting this article, if appropriate. Editing help is available. (August 2021)

|

(Learn how and when to remove this message)

Example of data collection in the biological sciences: Adélie penguins are identified and weighed each time they cross the automated weighbridge on their way to or from the sea.[1]

Data collection or data gathering is the process of gathering and measuring information on targeted variables in an established system, which then enables one to answer relevant questions and evaluate outcomes. Data collection is a research component in all study fields, including physical and social sciences, humanities,[2] and business. While methods vary by discipline, the emphasis on ensuring accurate and honest collection remains the same. The goal for all data collection is to capture evidence that allows data analysis to lead to the formulation of credible answers to the questions that have been posed.

Regardless of the field of or preference for defining data (quantitative or qualitative), accurate data collection is essential to maintain research integrity. The selection of appropriate data collection instruments (existing, modified, or newly developed) and delineated instructions for their correct use reduce the likelihood of errors.

## Methodology

[edit]

This article is missing information about experiment, sampling, measurement and preprocessing. Please expand the article to include this information. Further details may exist on the talk page. (July 2023)

|

See also: scientific method

Data collection and validation consist of four steps when it involves taking a census and seven steps when it involves sampling.[3]

A formal data collection process is necessary, as it ensures that the data gathered are both defined and accurate. This way, subsequent decisions based on arguments embodied in the findings are made using valid data.[4] The process provides both a baseline from which to measure and in certain cases an indication of what to improve.

### Tools

[edit]

#### Data collection system

[edit]

Main article: Data collection system

#### Data management platform

[edit]

Main article: Data management platform

Data management platforms (DMP) are centralized storage and analytical systems for data, mainly used in marketing. DMPs exist to compile and transform large amounts of demand and supply data into discernible information. Marketers may want to receive and utilize first, second and third-party data. DMPs enable this, because they are the aggregate system of DSPs (demand side platform) and SSPs (supply side platform). DMPs are integral for optimizing and future advertising campaigns.

### Data integrity issues

[edit]

The main reason for maintaining data integrity is to support the observation of errors in the data collection process. Those errors may be made intentionally (deliberate falsification) or non-intentionally (random or systematic errors).[5]

There are two approaches that may protect data integrity and secure scientific validity of study results:[6]

* Quality assurance – all actions carried out before data collection
* Quality control – all actions carried out during and after data collection

#### Quality assurance (QA)

[edit]

Further information: Quality assurance

QA's focus is prevention, which is primarily a cost-effective activity to protect the integrity of data collection. Standardization of protocol, with comprehensive and detailed procedure descriptions for data collection, are central for prevention. The risk of failing to identify problems and errors in the research process is often caused by poorly written guidelines. Listed are several examples of such failures:

* Uncertainty of timing, methods and identification of the responsible person
* Partial listing of items needed to be collected
* Vague description of data collection instruments instead of rigorous step-by-step instructions on administering tests
* Failure to recognize exact content and strategies for training and retraining staff members responsible for data collection
* Unclear instructions for using, making adjustments to, and calibrating data collection equipment
* No predetermined mechanism to document changes in procedures that occur during the investigation

#### User privacy issues

[edit]

There are serious concerns about the integrity of individual user data collected by cloud computing, because this data is transferred across countries that have different standards of protection for individual user data.[7] Information processing has advanced to the level where user data can now be used to predict what an individual is saying before they even speak.[8]

#### Quality control (QC)

[edit]

Further information: Quality control

Since QC actions occur during or after the data collection, all the details can be carefully documented. There is a necessity for a clearly defined communication structure as a precondition for establishing monitoring systems. Uncertainty about the flow of information is not recommended, as a poorly organized communication structure leads to lax monitoring and can also limit the opportunities for detecting errors. Quality control is also responsible for the identification of actions necessary for correcting faulty data collection practices and also minimizing such future occurrences. A team is more likely to not realize the necessity to perform these actions if their procedures are written vaguely and are not based on feedback or education.

Data collection problems that necessitate prompt action:

* Systematic errors
* Violation of protocol
* Fraud or scientific misconduct
* Errors in individual data items
* Individual staff or site performance problems
* Shadow effect

## See also

[edit]

* Controlled experiment
* Data acquisition
* Data curation
* Data management
* Observational study
* Sampling (statistics)
* Scientific data archiving
* Statistical survey
* Survey data collection
* Qualitative method
* Quantitative method
* Quantitative methods in criminology
* Data mining

## References

[edit]

1. ^ Lescroël, A. L.; Ballard, G.; Grémillet, D.; Authier, M.; Ainley, D. G. (2014). Descamps, Sébastien (ed.). "Antarctic Climate Change: Extreme Events Disrupt Plastic Phenotypic Response in Adélie Penguins". PLOS ONE. 9 (1): e85291. Bibcode:2014PLoSO...985291L. doi:10.1371/journal.pone.0085291. PMC 3906005. PMID 24489657.
2. ^ Vuong, Quan-Hoang; La, Viet-Phuong; Vuong, Thu-Trang; Ho, Manh-Toan; Nguyen, Hong-Kong T.; Nguyen, Viet-Ha; Pham, Hiep-Hung; Ho, Manh-Tung (September 25, 2018). "An open database of productivity in Vietnam's social sciences and humanities for public use". Scientific Data. 5: 180188. Bibcode:2018NatSD...580188V. doi:10.1038/sdata.2018.188. PMC 6154282. PMID 30251992.
3. ^ Ziafati Bafarasat, A. (2021) Collecting and validating data: A simple guide for researchers. Advance. Preprint.. https://doi.org/10.31124/advance.13637864.v1
4. ^ Data Collection and Analysis By Dr. Roger Sapsford, Victor Jupp ISBN 0-7619-5046-X
5. ^ Northern Illinois University (2005). "Data Collection". Responsible Conduct in Data Management. Retrieved June 8, 2019.
6. ^ Most, Marlene M.; Craddick, Shirley; Crawford, Staci; Redican, Susan; Rhodes, Donna; Rukenbrod, Fran; Laws, Reesa (October 2003). "Dietary quality assurance processes of the DASH-Sodium controlled diet study". Journal of the American Dietetic Association. 103 (10): 1339–1346\. doi:10.1016/s0002-8223(03)01080-0. PMID 14520254.
7. ^ Wang, Faye Fangfei (10 January 2014). Law of Electronic Commercial Transactions: Contemporary Issues in the EU, US and China. Routledge. p. 154\. ISBN 978-1-134-11522-8.
8. ^ "Data, not privacy, is the real danger". NBC News. 4 February 2019.

## External links

[edit]

Wikimedia Commons has media related to Data collection.

* All about data collection – TechTarget.com

* v
* t
* e

Data

* Acquisition
* Augmentation
* Analysis
* Anonymization
* Archaeology
* Big
* Cleansing
* Collection
* Compression
* Corruption
* Curation
* Deduplication
* Degradation
* De-identification
* Ecosystem
* Editing
* Engineering
* Erasure
* ETL/ELT
* Extract
* Transform
* Load
* Ethics
* Exhaust
* Exploration
* Farming
* Format management
* Fusion
* Governance
* Cooperatives
* Infrastructure
* Integration
* Integrity
* Library
* Lineage
* Loss
* Management
* Meta
* Migration
* Mining
* Philanthropy
* Pre-processing
* Preservation
* Processing
* Protection (privacy)
* Publishing
* Open data
* Recovery
* Reduction
* Redundancy
* Re-identification
* Remanence
* Rescue
* Retention
* Quality
* Science
* Scraping
* Scrubbing
* Security
* Sharing
* Stewardship
* Storage
* Structure
* Synchronization
* Topological data analysis
* Type
* Validation
* Warehouse
* Wrangling/munging

* v
* t
* e

Statistics

* Outline
* Index

Descriptive statistics

| Continuous data|

| Center|

* Mean
* Arithmetic
* Arithmetic-Geometric
* Contraharmonic
* Cubic
* Generalized/power
* Geometric
* Harmonic
* Heronian
* Heinz
* Lehmer
* Median
* Mode

|

Dispersion|

* Average absolute deviation
* Coefficient of variation
* Interquartile range
* Percentile
* Range
* Standard deviation
* Variance

Shape|

* Central limit theorem
* Moments
* Kurtosis
* L-moments
* Skewness

Count data|

* Index of dispersion

Summary tables|

* Contingency table
* Frequency distribution
* Grouped data

Dependence|

* Partial correlation
* Pearson product-moment correlation
* Rank correlation
* Kendall's τ
* Spearman's ρ
* Scatter plot

Graphics|

* Bar chart
* Biplot
* Box plot
* Control chart
* Correlogram
* Fan chart
* Forest plot
* Histogram
* Pie chart
* Q–Q plot
* Radar chart
* Run chart
* Scatter plot
* Stem-and-leaf display
* Violin plot

Data collection

| Study design|

* Effect size
* Missing data
* Optimal design
* Population
* Replication
* Sample size determination
* Statistic
* Statistical power

|

Survey methodology|

* Sampling
* Cluster
* Stratified
* Opinion poll
* Questionnaire
* Standard error

Controlled experiments|

* Blocking
* Factorial experiment
* Interaction
* Random assignment
* Randomized controlled trial
* Randomized experiment
* Scientific control

Adaptive designs|

* Adaptive clinical trial
* Stochastic approximation
* Up-and-down designs

Observational studies|

* Cohort study
* Cross-sectional study
* Natural experiment
* Quasi-experiment

Statistical inference

| Statistical theory|

* Population
* Statistic
* Probability distribution
* Sampling distribution
* Order statistic
* Empirical distribution
* Density estimation
* Statistical model
* Model specification
* Lp space
* Parameter
* location
* scale
* shape
* Parametric family
* Likelihood (monotone)
* Location–scale family
* Exponential family
* Completeness
* Sufficiency
* Statistical functional
* Bootstrap
* U
* V
* Optimal decision
* loss function
* Efficiency
* Statistical distance
* divergence
* Asymptotics
* Robustness

|

Frequentist inference|

| Point estimation|

* Estimating equations
* Maximum likelihood
* Method of moments
* M-estimator
* Minimum distance
* Unbiased estimators
* Mean-unbiased minimum-variance
* Rao–Blackwellization
* Lehmann–Scheffé theorem
* Median unbiased
* Plug-in

|

Interval estimation|

* Confidence interval
* Pivot
* Likelihood interval
* Prediction interval
* Tolerance interval
* Resampling
* Bootstrap
* Jackknife

Testing hypotheses|

* 1- & 2-tails
* Power
* Uniformly most powerful test
* Permutation test
* Randomization test
* Multiple comparisons

Parametric tests|

* Likelihood-ratio
* Score/Lagrange multiplier
* Wald

Specific tests|

* Z-test (normal)
* Student's t-test
* F-test

Goodness of fit|

* Chi-squared
* G-test
* Kolmogorov–Smirnov
* Anderson–Darling
* Lilliefors
* Jarque–Bera
* Normality (Shapiro–Wilk)
* Likelihood-ratio test
* Model selection
* Cross validation
* AIC
* BIC

Rank statistics|

* Sign
* Sample median
* Signed rank (Wilcoxon)
* Hodges–Lehmann estimator
* Rank sum (Mann–Whitney)
* Nonparametric anova
* 1-way (Kruskal–Wallis)
* 2-way (Friedman)
* Ordered alternative (Jonckheere–Terpstra)
* Van der Waerden test

Bayesian inference|

* Bayesian probability
* prior
* posterior
* Credible interval
* Bayes factor
* Bayesian estimator
* Maximum posterior estimator

* Correlation
* Regression analysis

| Correlation|

* Pearson product-moment
* Partial correlation
* Confounding variable
* Coefficient of determination

|

Regression analysis|

* Errors and residuals
* Regression validation
* Mixed effects models
* Simultaneous equations models
* Multivariate adaptive regression splines (MARS)

Linear regression|

* Simple linear regression
* Ordinary least squares
* General linear model
* Bayesian regression

Non-standard predictors|

* Nonlinear regression
* Nonparametric
* Semiparametric
* Isotonic
* Robust
* Homoscedasticity and Heteroscedasticity

Generalized linear model|

* Exponential families
* Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance|

* Analysis of variance (ANOVA, anova)
* Analysis of covariance
* Multivariate ANOVA
* Degrees of freedom

Categorical / multivariate / time-series / survival analysis

| Categorical|

* Cohen's kappa
* Contingency table
* Graphical model
* Log-linear model
* McNemar's test
* Cochran–Mantel–Haenszel statistics

|

Multivariate|

* Regression
* Manova
* Principal components
* Canonical correlation
* Discriminant analysis
* Cluster analysis
* Classification
* Structural equation model
* Factor analysis
* Multivariate distributions
* Elliptical distributions
* Normal

Time-series|

| General|

* Decomposition
* Trend
* Stationarity
* Seasonal adjustment
* Exponential smoothing
* Cointegration
* Structural break
* Granger causality

|

Specific tests|

* Dickey–Fuller
* Johansen
* Q-statistic (Ljung–Box)
* Durbin–Watson
* Breusch–Godfrey

Time domain|

* Autocorrelation (ACF)
* partial (PACF)
* Cross-correlation (XCF)
* ARMA model
* ARIMA model (Box–Jenkins)
* Autoregressive conditional heteroskedasticity (ARCH)
* Vector autoregression (VAR)

Frequency domain|

* Spectral density estimation
* Fourier analysis
* Least-squares spectral analysis
* Wavelet
* Whittle likelihood

Survival|

| Survival function|

* Kaplan–Meier estimator (product limit)
* Proportional hazards models
* Accelerated failure time (AFT) model
* First hitting time

|

Hazard function|

* Nelson–Aalen estimator

Test|

* Log-rank test

Applications

| Biostatistics|

* Bioinformatics
* Clinical trials / studies
* Epidemiology
* Medical statistics

|

Engineering statistics|

* Chemometrics
* Methods engineering
* Probabilistic design
* Process / quality control
* Reliability
* System identification

Social statistics|

* Actuarial science
* Census
* Crime statistics
* Demography
* Econometrics
* Jurimetrics
* National accounts
* Official statistics
* Population statistics
* Psychometrics

Spatial statistics|

* Cartography
* Environmental statistics
* Geographic information system
* Geostatistics
* Kriging

* Category
* Mathematics portal
* Commons
* WikiProject

Portals:

* Business and economics
* Science
* Mathematics
* Contents

Authority control databases: National |

* Germany
* Czech Republic

|

*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template