AMSTERDAM – Booking.com, a major player in the online travel sector, is doubling down on artificial intelligence to boost its software development capabilities.With over 500 million monthly visitors and a billion annual bookings, the company is focused on maintaining its competitive edge by accelerating software delivery and innovation. To achieve this, Booking.com is integrating AI across its engineering teams.

“We’re very excited about GenAI as it has the potential to multiply the impact of our engineering organization,” explains Amos Haviv from Booking.com’s Developer Experience team. “our vision is to give every developer the equivalent of a senior engineer sitting beside them to pair program and tackle problems,” adds Bruno Passos, Group Product Manager of the team responsible for spearheading their AI efforts.

Overcoming Initial Hurdles

The initial rollout of AI code assistants at Booking.com faced some challenges. Differing expectations between executives and developers created obstacles, with some executives having overly optimistic expectations, while developers were skeptical about the actual benefits of the tools.

“The whole world was talking about AI, and the numbers we were hearing were astronomical: claims of hundreds of thousands of hours saved for developers,” shares Passos. “But nothing was backing it up. So our initial goal was simply to understand what GenAI could actually do for us and figure out a way to measure the value it was delivering to the organization.”

Along with assessing the real impact of AI, Booking.com needed data to inform decisions about vendor selection and to guide training efforts to ensure triumphant adoption.

“We needed to understand how AI affected engineering velocity, satisfaction, and code quality,” says Passos. “Without that insight, we wouldn’t be able to confidently talk about the ROI of Booking’s investment in AI with the rest of the business.We also wouldn’t know for sure whether we shoudl be driving more adoption for a specific tool or use case.”

“We’re very excited about GenAI because it has the potential to multiply the impact of our engineering organization.”

Data-Driven Implementation

To accurately measure the adoption and impact of AI tools, Booking.com implemented a data-driven approach. By analyzing various productivity signals, the company gained valuable insights into how AI was affecting its software development teams.

Key findings included:

• Developers who used AI daily demonstrated 16% higher code throughput compared to those who did not.
• AI users were able to save time on routine tasks, allowing them to concentrate on more critical work.
• Developer satisfaction with AI tools increased by 15 points over six months, thanks to internal training initiatives.

These findings were based on both qualitative and quantitative data. By combining direct feedback from developers with longitudinal analyses of productivity signals, Booking.com gained a extensive understanding of the impact of AI across the organization.

Zane Wright, Senior Product Manager, shares: “We’ve been able to use data to make tactical and strategic decisions on where to invest further in our GenAI program. Decisions like wich vendors we should be going for, where we should be looking further to assess deeper impact on our company, and how we should be structuring our programs to best impact developers moving forward.”

One surprising insight was that many developers were hesitant to adopt AI tools due to skepticism. To address this, Booking.com increased its investment in developer training. “We realized that education would be just as significant to increasing adoption as improvements to the technology itself,” shares Passos.

Expanding AI Adoption

With data confirming the positive impact of AI on developer productivity, Booking.com is now focused on achieving 100% adoption and encouraging more frequent use of AI tools.

“The data showed us that developers were most effective if they used AI on a daily basis, or at least twelve days per month. So we set out on a mission to figure out how to drive not only adoption, but widespread daily usage,” says Passos.

Booking.com has implemented several strategies to achieve this:

• Segmentation and targeted outreach: Identifying developer segments that are getting more or less value from AI tools and providing tailored support.
• Education and hands-on experiences: Running workshops focused on AI education and applying AI to solve real-world problems.
• Continuous communication: Regularly sharing updates on new AI features and capabilities.
• Office hours and training: Providing developers with direct support and training on AI tools and best practices.

As an inevitable result of these efforts, Booking.com has seen a significant increase in developer adoption and utilization of AI tools, leading to further productivity gains. Teams that have fully adopted AI show 30% higher throughput then those that have not.

Looking ahead, Booking.com plans to continue integrating AI tools into its software development lifecycle to maintain its competitive position.