Don't Go Chasing Waterfalls: How Agile Makes Complex Web Projects Enjoyable

The Stress of Waterfall Planning in Web CMS Projects 

When you have a giant project ahead of you, even thinking about it can be stressful. If you’re doing the work, there is tremendous pressure to estimate accurately, follow best practices, and finish on time. If you are managing the work, you have the stress of keeping the timeline and budget on track. And if you’re sponsoring the work, your neck is on the line to ensure it produces the results you promised other stakeholders and executives when your budget was approved. 

There’s no need to dread large, complex projects. How you approach your project and organize the work affects whether the experience is a rewarding challenge or an agonizing journey. Waterfall projects pit people against one another in servitude to a rigid project plan. Agile projects still incorporate planning but focus more on collaboration and outcomes. Let’s break down how the different approaches affect the experience of the people working on or managing the project. 

Waterfall Projects: An Exercise in Optimism 

The allure of waterfall projects is that they look great on paper. Each step is itemized as a line item, with planned durations and deadlines. All major deliverables are mapped to specific dates. Contingency budgets are (hopefully) held aside to handle any surprises. All this goes a long way toward increasing project manager and stakeholder confidence that the project will be successful while alleviating concerns about the timeline, scope, and budget from the onset. 

Initial Optimism and Subsequent Anxiety in Waterfall Projects 

The post-kickoff period is the high point of the waterfall experience. In these early stages, anyone not performing the work (managers, stakeholders) feels great because the plan seems bulletproof. Assuming your implementation team has been consulted when creating the scope and estimates, they'll feel pretty good too. There’s a palpable electricity that courses through everyone’s veins after a good kickoff, and it fuels this tragic optimism—it happens almost every time. 

Optimism ebbs and flows at different points, but sooner or later, harsh reality sets in, moods change, and project anxiety increases. Here are some reasons waterfall projects can feel like such an emotional roller coaster: 

Project Teams: Burdened by Inaccurate Estimates in Waterfall Methodology 

People are not great at estimating complex work. In waterfall CMS projects, estimates provided by developers are often highly inaccurate. The estimates are laden with assumptions and produced prior to the team digging into the business logic and code and seeing the actual complexity. Despite this, these initial estimates are converted into a holy Gantt chart to set concrete expectations for the project timeline and cost. 

From this point forward, the team is under the gun to deliver each project piece within their estimates. Inevitably, estimates prove inaccurate, and certain tasks take longer than expected, which on bigger projects can create a snowball effect. 

Once the team gets behind, their options are: 

• Working overtime to hit the deadlines. 
• Cutting corners to catch up. 
• Not logging all their time (to avoid getting harassed about the budget). 

Shifting deadlines is also a possibility, but when poor estimation is blamed, the team that provided the estimates usually takes the brunt of it. 

Pressure to deliver within a timeline that feels increasingly unreasonable is stressful and creates unnecessary tension between the implementation team and the project managers or stakeholders. Teams get resistant to even the smallest change requests, as any extra work will only put them farther behind. The early optimism is replaced by jaded skepticism and a burnt-out project team that can’t wait for the project to be over. 

Project Managers: Caught Between Plans and Reality in Waterfall Projects 

Project managers are expected to ensure the project is completed on time and on budget, which has never made sense to me since they have minimal influence over it. They aren’t the people doing the work, so they can’t control the time it takes to complete things. They usually aren’t the ones pricing the work or defining the scope, so they’re at the mercy of whatever contractual obligations exist. 

Good project managers pad dev team estimates, build in a contingency budget, and develop a reasonable project plan. They will manage expectations and ensure a clear flow of communication between the team and stakeholders. The plan should work, assuming everyone sticks to the plan. So, the best they can do is to protect the plan by shutting down anything that might change it—including missed deadlines, requirement or scope changes, and getting stakeholders to adhere to the deadlines for reviewing work and giving feedback. 

But the dev team will plead for more time. And the stakeholders (or UX designers) will want to change requirements. Because the timeline and budget are the measure of project managerial success, they are incentivized to stick to the plan at all costs, even if a different course of action would produce better business results. 

When things don’t go to plan, project managers are put in the stressful situation of telling somebody they can’t get what they want. They must either pressure the team to work harder and faster to catch up, tell their client something is out of scope, or tell stakeholders that the project will cost more money or take longer to deliver—expectations they had set and are accountable for protecting. Either way, they are in a position where it can be perceived as picking sides. 

Stakeholders: Struggling for Control and Satisfaction in Waterfall Projects 

In waterfall projects, as a client or business stakeholder, you are asked for feedback and approval at specific milestones, which puts you under pressure to speak then or forever hold your peace. Approval for design is requested months before development. This means decisions around functionality are often demanded while mockups or prototypes are in a pseudo-functional state with sample (not real or dynamic) data used as placeholders. With key decisions at stake and limited ability to course correct later in the project, it’s not uncommon to want extra days to review proposed designs with your team or to get executive sign-off from higher-ups who are harder to get a hold of. The approvals and feedback may be necessary but can jeopardize the project schedule, which you have likely also committed to with your project sponsors or executive team. 

Once development begins, work is handed off in batches for your approval. At this point, concerns will almost certainly arise. Some aspects of the design or functionality that seemed right during the design and requirements phase don’t make sense now that you and other stakeholders see them built out and can test them with real content. 

At this point, changes to design and functionality are usually considered a scope change. Your frustration will increase when seemingly small change requests are met with resistance from the project team, an add-cost change order, or a timeline shift. 

Depending on your organization and how you budget for web projects, the feasibility of going back to ask for more funding or push back deadlines may be tough. To keep your project on schedule and protect your project budget, the site changes you wanted will get kicked to a “phase 2” project. Spoiler: phase 2 rarely happens unless phase 1 shows a tangible return on investment. 

Waterfall Methodology: A Recipe for Conflict in Web Projects 

Even with contingency and some revision cycles built into your workflow, the waterfall project model is conducive to conflict between all parties. 

The issue with this approach is that each group involved in the project is trying to protect different things, and the structure of the engagement is one where everyone’s objectives are at odds. The folks designing and building things just want to produce good design that benefits users and write high-quality, performant code. The project managers want to channel any divine power that will keep the plan on track and avoid change or disruption. The stakeholders want to get their money’s worth and be able to satisfy their executive team and their customers with the work they sponsored. 

Sometimes, the stars align, and things go fine. But usually, one of those three groups feels like they got the raw end of the deal. The solution to this dilemma isn’t better planning; it’s shifting to an agile project management model that is better aligned with the messiness of complex web projects. 

The Agile Experience: A Focus on Problem Solving 

Because waterfall projects “work well on paper,” this approach feels great at the onset for project managers, stakeholders, and any OCD-natured people on the project. In contrast, the agile approach can feel uncomfortable early on when some details are still fuzzy. 

Agile Projects: Embracing Change and Flexibility in Web Development 

Despite the misconception, agile projects still incorporate planning and are driven by budget and timeline. Some upfront planning is necessary to capture outcomes, outline key features and functionality, deadlines, and release strategy. The difference is that with an agile project, we embrace the expectation that the plan will evolve and requirements will change. Because of this, less effort is spent trying to perfect granular estimates and functional specifications up front, and more energy is devoted to decision-making along the way. 

While a rigid and detailed waterfall project plan can easily unravel as the project progresses, with an agile approach what starts as a rough plan increases in accuracy and clarity each step of the way. I think this is critical to improving the shared experience of everyone on the project. 

A plan that evolves continuously based on the best interest of the project goals and is shaped by the collective insights of the people closest to the business needs, user needs, and technology, is less prone to conflict and more conducive to collaboration. 

Reduced Stress for Project Teams in Agile Methodology 

Embracing change extends to estimates as well. Rather than building out extensive requirements for full features upfront, teams work with stakeholders to discuss scenarios for ways that users should be able to interact with the site, and then break web components and features into smaller subsets of functionality (usually referred to as stories). 

Early on, each story is roughly estimated and prioritized based on value to the customer, but rather than committing to these estimates with an oath signed in developer blood, estimates are reviewed and updated regularly as the time to work on each piece draws near. The same goes for functional requirements. The flexibility to revisit estimates and requirements accomplishes a few things: 

• Upfront estimates, created when the team doesn’t know what they don’t know, are no longer weaponized against them. 
• Estimates revised based on the latest information and learnings are more reliable than estimates created months earlier. This might bring good news or bad news, but it’s healthier to constantly confront project reality than to prolong tragic optimism. 

• The flexibility to adjust functional requirements shifts conversations between developers and stakeholders away from contractual disputes about what was agreed to and toward conversations about what makes the most sense to do, given the latest insights. 

Collaborative Decision-Making Over Handoffs in Agile Projects 

Building a project plan that does these things demands more than a mindset shift. The structure of an agile project is inherently different from the structure of a waterfall project plan. Here are the characteristics of a well-planned agile project that are necessary in order for flexibility to coexist with budgets and timelines: 

• Teams work cross-functionally, replacing handoffs with conversations, and ensuring that important decisions are not made in a vacuum. 
• Features are built out incrementally, with potentially releasable pieces of functionality turned over for testing and review continuously. Rather than fully building out each feature to spec before moving on to the next, an incremental approach creates opportunities for teams and stakeholders to negotiate and make trade-offs about what the next most valuable thing to work on is. 

Agile Project Management: Facilitating Change and Improving Outcomes 

On the surface, the notion that estimates and requirements are revisited continually throughout a project sounds like a project manager’s nightmare, but it’s actually a boon in disguise. 

In agile projects, outcomes are prioritized over sticking to the plan, and value delivered is the measure of project success. With outcome-driven goals, the incentive for project managers to discourage change goes away. The role of the project manager fundamentally shifts from defending the plan to helping teams navigate change and facilitating conversations about options and impact on budget and timeline. Here are some ways teams benefit from this approach: 

• By analyzing the pace of work completion, agile project managers can predict threats to the timeline and budget and proactively explore options to deal with the risks. 
• Because requirements can evolve, project managers no longer get stuck in scope disputes between stakeholders and the implementation team. Instead, they facilitate conversations about what options exist given the reality of available budget and timeline constraints. 
• Estimates being more reliable in this model and work being delivered in regular demo-able increments, the project manager can spend less time pestering the team about when work will be done and more time removing obstacles and helping the team work effectively together.

More Options for Stakeholders in Agile Projects 

The problem with waterfall projects is that when things don’t go to plan, something’s got to give. The team works overtime, which diminishes quality. Or the timeline shifts. Or somebody is paying extra. In any scenario, it’s contentious. Agile projects encourage collaboration, reprioritization, and replanning with stakeholders throughout the project. Here’s how that benefits the stakeholder experience: 

• Reprioritization ensures that you have a voice in what the next most important things are for the team to be working on, so that you don’t end up in a situation where the project is held hostage and can’t be released because the most important features haven’t been worked on yet. Instead, you have the option to release earlier with the most important functionality and follow up in subsequent code releases with additional enhancements. 
• It's more collaborative since you can continually inspect the work in progress and offer feedback so course correction can occur as needed during the project. This means less pressure to commit to decisions early on in the project and then be handcuffed to those decisions once you see working features. 
• Replanning occurs so that when things change, everyone works on considering options and negotiating the best path forward that protects project outcomes. Instead of contention with the project team and feeling like painful sacrifices are the only path to launch, stakeholders and team members have the freedom to collaborate on solutions and adapt the plan in whatever way best serves the business and customer needs within the confines of timeline and budget. 

"That’s Not Really Agile, Bro” 

Some haters out there might be in arms, calling out aspects of the approach outlined above that are not truly agile. But agile is not black and white; it’s subject to interpretation, and many people practice it differently. To me, agility is about flexibility. It’s empowering teams to avoid the pitfalls that come with sticking to a predefined plan, by changing the plan when it makes sense to. 

I don’t think it makes sense to spend months stressed out about your project and sitting in contentious meetings with stakeholders who are being told they can’t have what they need because they previously signed off on a direction. Or seeing team members burnt out about impossible deadlines because some magic spreadsheet formula took early human guesstimates and plotted them into a schedule that looks increasingly unrealistic each day. 

Call it what you will, but your project plan can bring out the best in people or the worst. It’s my belief that flexible, agile project plans feel a whole lot more satisfying to work on than following a rigid plan, created in some absurd project prequel, that now everyone’s expected to blindly follow.