The increasing in uncertainty drove each era of software development practices
In Wild West To Agile - Jim Highsmith, Highsmith uses Snowden's Cynefin framework to trace software engineering's evolution across four eras:
Wild West (1966–1979): The environment was "obvious/complicated." Problems were relatively straightforward, do necessary things faster. Software was arcane, but the value proposition was simple: productivity and cost reduction.
The belief was that the world is predictable, so company cultures tended to be inflexible, hierarchical, command-control and focused on planning and executing. If the plan wasn't working, than something was wrong with the execution.
Structured (1980–1989): Still "complicated," edging toward complexity. The response was imposing order through structured methods and monumental methodologies, trying to counteract uncertainty by demanding more certainty.
Roots of Agile (1990–2000): The world shifted to "complex." This sparked rebellion against prescriptive methodologies that couldn't deliver flexibility and speed. Highsmith's epiphany came from mountain climbing: climbers adapt to the reality of a mountain, yet software projects expected reality to adapt to plans. See The correct response to complexity is probe, sense, respond.
Agile (2001–now): The environment became "chaotic/disorder." The old mindset, that you could develop a plan and execute with minimal deviation, that processes could drive out variation, that people were cogs, simply stopped working. Understanding that Software development is complex, not complicated became critical, which is why There are no best practices in complex systems.