[DATA][ENG.] 1. Why EDA(Exporatory Data Analsis)?

Last updated on 

Taking Time to Understand the Data Before Using It

When starting data analysis, many people immediately think about models.

However, in my experience, about 70% of a model’s performance is already determined during EDA.

EDA is the process of understanding what kind of data this is, whether it can be trusted, and what questions can be asked of it.

Why Is EDA Necessary?

When analysis starts without EDA, the following problems often occur:

  • Training a model without realizing the data has many missing values
  • Using a column with constant values as a feature
  • Training directly on data with distorted distributions

EDA prevents these mistakes early on.

EDA is closer to having a conversation with the data than to “analysis.”

Core Questions of EDA

When doing EDA, I always start by asking the following questions.

1. What does this data look like?

  • What does a row represent?
  • What does each column mean?
  • Is there a notion of time? (event time, snapshot time)

Example:

  • row = one customer
  • column = customer behavior / attributes

2. Is the data sufficient?

  • What is the total number of samples?
  • What is the class distribution?
  • Is the data concentrated in a specific time period?
df.shape
df['label'].value_counts(normalize=True)

Here, we judge whether the data size is sufficient and whether class imbalance is severe.

3. What about missing values and outliers?

This is the first thing to check in EDA.

df.isnull().mean().sort_values(ascending=False)
  • Why do some columns have many missing values?
  • Are they “meaningful zeros” or truly “missing values”?
  • Do missing values occur only in specific ranges?

Missing values are not just a problem—they are clues about the data generation process.

4. Are the distributions reasonable?

For numerical data, always inspect the distribution.

df['price'].hist(bins=50)

Checkpoint:

You must be able to trust the distribution before trusting the model.

5. What are the relationships between variables?

The essence of EDA lies in exploring relationships.

  • Feature ↔ target relationships
  • Feature ↔ feature correlations
df.corr(numeric_only=True)

Insights gained here include:

  • Removing useless variables
  • Discovering redundant features
  • Finding signals that can be explained by simple rules

Visualization Is the Language of EDA

EDA is a phase with more pictures than numbers.

Commonly used visualizations:

  • Histograms (distribution)
  • Box plots (outliers)
  • Scatter plots (relationships)
  • Time-series line charts

If you find it hard to explain EDA results in words, it likely means you don’t fully understand the data yet.

Practical EDA Checklist

Below is an EDA checklist I use almost every time in practice:

  • Understand the data schema
  • Clarify the meaning of rows and columns
  • Check missing value ratios
  • Identify outliers
  • Inspect target distribution
  • Visualize key feature distributions
  • Explore feature–target relationships
  • Check for potential data leakage
  • Verify whether time-based splits are possible

EDA Does Not End After One Pass

One important point:

EDA is not something you do once before modeling and then forget.

  • Create new features → EDA again
  • Model performance is poor → EDA again
  • Data changes → EDA again

EDA ↔ Modeling ↔ EDA

The more this cycle repeats, the better the analysis becomes.

Conclusion

EDA is the stage that prevents the most mistakes.

If your model isn’t performing well,

before tuning hyperparameters, ask this question first:

“Do I truly understand this data?”

EDA is the process that answers that question.