The story of how migrating Tableau to Django led to a complete rebuild due to design flaws

Design is 90% of the work

Introduction

I once rebuilt a dashboard that was originally created in Tableau as a web system using Django.

Right before the release, this is what happened:

• It took over 10 seconds to display after clicking a button

• The screen froze every time a filter was changed

• The numbers were slightly different every time the same screen was opened

"It works, sort of. But we can't put this into production, can we?"

I spent a week worrying about whether I could just fix it as is.

I tried optimizing queries, adding indexes, and implementing caching.

But it didn't improve.

That's when I finally realized:

This isn't an implementation issue, it's a design issue

→ I decided to rebuild everything from scratch.

Why did I migrate?

・Tableau's response was slow
・Complex UI control was not possible
・It was difficult to integrate into business workflows

"If we make it a web app, it will be solved."

Thinking that, I rebuilt it in Django.

The first mistake I made

Replicating Tableau screens exactly as they were

What happened

I ended up with something that "worked but was unusable."

SELECT
    s.store_id,
    SUM(s.amount),
    COUNT(oi.order_id)
FROM sales s
JOIN order_items oi ON s.order_id = oi.order_id
JOIN customers c ON s.customer_id = c.id
WHERE s.sales_date BETWEEN :from AND :to
GROUP BY s.store_id;

I was executing this every time a screen action was performed.

Why it was unusable

・Changing filters → 10-second wait every time
・Concurrent access → Timeouts
・SQL different for every screen → Fixes spread horizontally
・Numbers misaligned → Unable to trace the cause

"It displays, but it's not usable for business."

Why it failed

I didn't do any data design

What I did to redo it

I threw away the screens and rebuilt it starting from the data.

1. Data mart design

・Determine granularity (1 store × 1 day)
・Eliminate the need for JOINs
・Create pre-aggregated tables

CREATE TABLE sales_daily (
    store_id INT,
    sales_date DATE,
    total_amount DECIMAL,
    order_count INT,
    PRIMARY KEY (store_id, sales_date)
);

2. Pre-aggregation via ETL

INSERT INTO sales_daily
SELECT
    s.store_id,
    s.sales_date,
    SUM(s.amount),
    COUNT(*)
FROM sales s
GROUP BY s.store_id, s.sales_date;

3. Simplification on the Django side

SELECT *
FROM sales_daily
WHERE sales_date BETWEEN :from AND :to;

4. Separation of roles

ETL: Calculation
DB: Storage
Django: Retrieval
Screen: Display

Results

・Response: 10 seconds → 0.5 seconds
・SQL: Complex → Simple
・Fixes: Completed in one place

The biggest lesson learned

BI is determined by data design, not appearance

Summary

This is what went wrong this time.

What I tried to replicate

The correct order is as follows.

1. Data design
2. Data flow
3. Backend
4. UI

Conclusion

If you get the design wrong, you will break in the same place even if you change tools.

Related articles

・6 common Tableau accidents
・6 accidents caused by poor data mart design
・6 practical SQL techniques

For those who are also moving BI to the web, please start by thinking about data design first.

If you found this helpful, a like would be encouraging.