I recently had the chance to present an updated version of my 7 wastes of data production talk at DataEngBytes Melbourne 2023. I think the talk was stronger this time around and I really appreciated all the great feedback from the audience.
Thanks to Peter Hanssens and the DEB crew for having me as part of an impressive speaker lineup and for putting on a great event.
Check out the video below and the slides.
For the earlier versions, see the original 7 wastes post and the 2021 LAST Conference version Data mesh: a lean perspective.
Outline
There’s a lot of ground to cover in 30 minutes with 7 wastes from both run and build lenses, plus 5 lean principles to address the waste. I’ll leave the summary here and encourage you to watch the video or read the slides if you want to know more.
| Waste | Run | Build |
| Overproduction | Unused products | Unused products |
| Inventory | Stored or processed data not used | Development work in progress |
| Overprocessing | Correcting poor quality data | Working with untrusted data |
| Transportation | Replication without reproducibility | Handoffs between teams |
| Motion | Manual intervention or finishing | Context switching |
| Waiting | Delays in taking action on business events | Delays due to handoffs or feedback lead time |
| Defects | Defects introduced into data at any point | Defects introduced into processing code |