Generate random data based on existing data

qwertz :

is there a way in python to generate random data based on the distribution of the alreday existing data?

Here are the statistical parameters of my dataset:

Data
count   209.000000
mean    1.280144
std     0.374602
min     0.880000
25%     1.060000
50%     1.150000
75%     1.400000
max     4.140000

as it is no normal distribution it is not possible to do it with np.random.normal. Any Ideas?

Thank you.

Edit: Performing KDE:

from sklearn.neighbors import KernelDensity
# Gaussian KDE
kde = KernelDensity(kernel='gaussian', bandwidth=0.525566).fit(data['y'].to_numpy().reshape(-1, 1))
sns.distplot(kde.sample(2400))

Peter O. :

In general, real-world data doesn't exactly follow a "nice" distribution like the normal or Weibull distributions.

Similarly to machine learning, there are generally two steps to sampling from a distribution of data points:

• Fit a data model to the data.

• Then, predict a new data point based on that model, with the help of randomness.

There are several ways to estimate the distribution of data and sample from that estimate:

• Kernel density estimation.
• Gaussian mixture models.
• Histograms.
• Regression models.
• Other machine learning models.

In addition, methods such as maximum likelihood estimation make it possible to fit a known distribution (such as the normal distribution) to data, but the estimated distribution is generally rougher than with kernel density estimation or other machine learning models.

See also my section "Random Numbers from a Distribution of Data Points".

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=301615&siteId=1

Generate random data based on existing data

[Generate data] random walk

How to: How to Generate XPO Business Classes for Existing Data Tables: generating XPO existing traffic class data table

Create data code - generate random numbers

How to quickly generate random data in Python?

ArcGis Python modifies field information based on existing data

How to generate random real test data for ClickHouse table

[ClickHouse series] How to use ClickHouse to generate random test data

R language recodes data or calculates and generates new data columns from existing data: use R built-in functions to recode data and generate new data columns (data.frame)

Generate the corresponding models model layer file from the existing data table in Django

importing existing user data with impdp

Based HttpRunner, swagger parsed data frame to quickly generate an interface test

[Translation] Generate grid data based on PostGIS3.1

Create new columns in a data frame based on an existing numeric column, a list of strings as column names and a list of tuples as values

Generate maven pom.xml based on the existing project jar file

[C++] Generate random numbers and write them into files, and read data from files for sorting

[Sklearn] Data classification prediction based on random forest algorithm (Excel can directly replace data)

Generate test data with ChatGPT

Generating random data

random access to data sqlserver

Data Augmentation - Random Rotation

Go reads the existing cached chan data

MySQL partitions existing tables in data tables

Partial solution of existing IIoT data management system

Integrate existing data - Data read from the standard text file

Js assignment after changing the existing data would modify the original data

SDK big data accurately labels existing data resources

Data enhancement script: Use positive samples and corresponding xml files to generate negative samples in batches through random cropping (bypassing all target boxes)

Based HttpRunner, analytical swagger data, quickly generate interface testing framework - optimized articles

How to use Python to automatically generate rainfall statistical analysis reports based on data