An agile approach to Data Science

Most Professional Services engagements with OpenText™ follow a traditional design, built, test and deploy project methodology.  OpenText software is well suited for the waterfall project…

Marc St-Pierre  profile picture
Marc St-Pierre

October 04, 20225 minute read

Most Professional Services engagements with OpenText™ follow a traditional design, built, test and deploy project methodology.  OpenText software is well suited for the waterfall project model.  A notable exception is OpenText™ Magellan™ and our Data Science projects.  In these cases, customers can expect an approach which simply adds refinement iterations to the build phase or otherwise, and full adoption of a more agile-like methodology, named OpenText Data Science Methodology.

Any data science project requires 3 important ingredients: Data, People and Technology.   With any element missing or being deficient, a data science project will be doomed, or even fail to get off the ground.  Finding the required data and the subject matter expertise is a complex exercise.  As this step may need to be tested and repeated many times over the lifetime of the project, a data science project must be approached with an agile or agile-like methodology.

What is Agile software development?

The History of Agile

A recent report by Wellingtone claims that only 29% of the organizations complete most or all of their projects on time, only 43% of the organizations complete most or all of their projects on budget and only 40% of organizations deliver the full benefits of most of their projects. Software projects are challenging for many reasons

In 2001, a group of thought leaders met at a ski resort in the Wasatch mountains, in Utah, to talk about the challenges of software development projects and how these projects could be more successful and efficient. They documented 4 key values in the Agile Manifesto to help project managers and developers build better software faster. 

The Agile Manifesto’s four key values:

1. Individuals and interactions

The Agile approach states that projects should be built around motivated individuals. The team can be motivated by a clear product vision and mission statement. It is crucial that the team remains focused on completing the tasks while working on the project. Team members perform best when they feel their contributions are important, valued and trusted. Face-to-face conversations are the most effective way to communicate. Communication via email is not as effective as it might take several email exchanges to sort out a simple issue that could be addressed with a short face-to-face conversation. 

2. Working software

According to the Agile Manifesto, product improvement is the main indicator of progress. Delivering valuable software to stakeholders early and frequently is more important than demonstrating progress through status reports. Having frequent releases of working software allows stakeholders to provide valuable feedback and lowers the risk of not meeting the needs of customers. 

3. Customer collaboration

Regular communication between the customer and developers is another critical key to building valuable software. Overthinking and over analyzing the requirements at the beginning of the project can lead to analysis paralysis and missed opportunities.

In Agile, the team makes decisions relatively quickly based on available information and adjusts the plan during the project based on regular communication between developers and the customer. The business value of features is determined by the customer, while the effort required to develop these features is provided by the developers.  This approach ensures that the team is always working on the right tasks and the end result fulfills the customer’s needs.

4. Responding to change

Agile means the ability to move quickly and easily. It also refers to flexibility and openness to change. The Agile philosophy acknowledges the fact that in our world things change constantly. According to Heraclitus and Plato, change is the only constant in life. 

Agile accepts and welcomes the lack of certainty and follows informal processes when requirements change. An Agile team must be able to react quickly to changing requirements, environments and shifting trends. 

When should you use Agile?

The Agile approach helps to deal with rapidly changing environments, uncertainty, complex solutions, emerging technologies, and ambiguous requirements. While Agile doesn’t eliminate uncertainty, it reduces risks and promises better outcomes in unpredictable and complex scenarios by delivering high-value products early and frequently and reviewing the result with the customer on a regular basis. 

OpenText expands this Agile approach into our Data Science Project methodology, adding focus to key areas:

  • Business understanding
  • Data understanding
  • Data preparation
  • Modeling
  • Evaluation
  • Deployment
  • And repeat..
OpenText Data Science Methodology

In our next blog, we will look more in-depth at the elements of the OpenText methodology and how it’s applied in engagements like Idea to Insight in 30 days.

Until then, discover how OpenText helps organizations become more agile by implementing business ready AI and Analytics solutions and providing Data Science, NLP and Knowledge Management services to solve today’s complex business challenges.  Visit AI & Analytics Services | OpenText for more information.


Contributor: George Berlak is the AI and Analytics Solution Development Manager for OpenText Professional Services. He is a certified Artificial Intelligence, Analytics and Data Science professional and leads a team of architects and engineers to design and build strategic AI, Analytics and Natural Language Processing solutions.

Share this post

Share this post to x. Share to linkedin. Mail to
Marc St-Pierre avatar image

Marc St-Pierre

Marc leads the OpenText Global Consulting Practice for Cybersecurity which delivers Risk & Compliance Advisory, Digital Forensics & Incident Response and various Managed Security Services. His mission is to promote Cyber Resilience and provide business partners with advice, guidance and assistance to achieve Digital Resilience & Trust. In his 15 years with OpenText, he has developed teams and built solutions in areas of Artificial Intelligence, LegalTech, Linguistics & Translation and now Cybersecurity. He has lectured on semantic technologies and lead growth of OpenText with innovations such as Ai-Augmented Voice of the Customer, Magellan Search+ and Managed Extended Detection & Response.

See all posts

More from the author

Unmasking the enemy!

Unmasking the enemy!

Behind the scenes of a successful threat hunt against a ransomware group

December 20, 2024 5 minute read

OpenText MxDR protecting the endpoint

OpenText MxDR protecting the endpoint

Latest EDR news from OpenText™ Cybersecurity Services

December 03, 2024 3 minute read

Enhancing Security Operations with OpenText Cybersecurity Services

Enhancing Security Operations with OpenText Cybersecurity Services

In today’s hyper-connected digital landscape, the proliferation of devices and applications has expanded the attack surface for potential cybersecurity threats. Cyberattacks are on track to…

July 19, 2024 4 minute read

Stay in the loop!

Get our most popular content delivered monthly to your inbox.