A Survey of Data Science Methodologies

On today’s show, Iñigo Martinez, a Ph.D. student at the University of Navarra who also has an affiliation with Vicomtech Foundation joins us. Iñigo research focuses on machine learning methods, clustering, classification for time series analysis, and anomaly detection. He joins us to discuss the findings of his survey that investigated success factors in data science projects.

Iñigo started by discussing the plethora of challenges data people face when implementing theoretical concepts in data science projects. While data science and software engineering methodologies may have some intersections, Iñigo pointed out some peculiarities of data science projects. He shared the stand-out methodology used for data science projects.

Iñigo spoke in depth about the survey he performed. The participants were a sample of 237 data practitioners. He shared the profile of the participants and the types of questions asked. He also revealed the methodologies participants mostly used. He shared his thoughts on why other methodologies are less adopted in the field. He then revealed what the success of a data science project means to the participants.

Iñigo probed into the factors that affect the success of a data science project. He shared the top success factors from the survey results and others that were shockingly less important to the participants. He also discussed other interesting results from the survey.

Concluding, Iñigo shared some best practices when conducting data science projects structured into three areas: project management, team management, and data management. You can follow Iñigo on Twitter @Inigoml or learn more about paper results here.