Helping a UK startup build the world’s largest event search engine.

Daybees is a search engine that aggregates millions of events happening across the UK.

Daybees is a UK startup that embarked on a mission to build the world’s largest event database and search engine. At its launch in 2014 it offered information about more than 30 million events from over 10 thousand sources. It was also featured in BBC, Wired, The New York Times, and other media.

London / UK
Web, iOS, Android



The founder of Daybees was frustrated with the availability of event information on the web. He envisioned an application aggregating all events, in every category and location - not just the mainstream ticketed events happening in major cities. It wasn’t just about finding events, but also incorporating them to an individual schedule with the help of the Daybees calendar. Adding, editing and sharing events, buying tickets, getting maps, avoiding clashes and finding out what else is happening around another event: Daybees aimed to help users build a calendar of events they could share with family, friends, and colleagues.


Daybees’ visionary mission set multiple challenges for the technical team. The first one was deciding how we would build and grow our event database to achieve maximum event coverage. We considered solutions such as Internet crawling, third party feeds, and user-generated content. Second, we had to process all the data aggregated from numerous sources to provide users with high-quality information. That included duplicate event detection, event merging and handling repetitive events which came in various combinations. Next, we had to serve all event data on a fast and intuitive web and mobile interface to ensure that Daybees users could find events they were interested in easily. Finally, we required a solid infrastructure with enough power to run our applications and serve the data processing work happening in the background.


After initial planning, the team made the following strategic decisions:

  • Programming language & framework - we chose Python and Django as a scalable web framework that ships easy- installable packages for creating APIs and a customized CMS.
  • Data collection and processing - we decided to rely on crawlers as the primary source of event data. That required writing a bespoke crawling script for each website and building a common crawler engine to run these scripts, process incoming data, and report on the results and system status.
  • Search engine - we chose Elasticsearch for the job because it provided us with indexing options that fit our requirements perfectly.
  • API - we decided to introduce an interface for web & mobile applications in the form of an abstraction layer over the database and search engine which would decouple them from web and mobile applications.

The next step was gathering two teams responsible for different areas of the system:

  • Design and development team composed of a CTO, a designer, and five developers - all focused on building the web and mobile apps with all their features,
  • Crawler team composed of four crawler engineers and a moderator - focused on building and maintaining a high- quality database of event data.

Both teams worked closely together for over a year to build, maintain, and grow Daybees. After the launch, three apps were running on the following infrastructure:

4 web application servers (behind a load balancer to ensure that more servers could be added easily in case of an increase of concurrent requests and connections),

3 database servers (master + slave + load balancing server),

4 Elasticsearch servers (as a scalable cluster for search queries and indexing data),

1 queue server (for long running tasks),

1 crawler server (where multi-threaded crawling and data postprocessing was happening).

  • Product design
  • Full-stack web and mobile development
  • API development
  • Search engine
  • Crawlers and aggregators
  • Python / Django
  • iOS / Swift
  • Android



most popular free app in the UK AppStore (Jan ‘14)


raised in funding


events in the database

11,113 URLs

crawled by 2,085 crawlers


people in the dedicated Daybees team


crawled event locations

Need top engineers for your team?

Got a project on your mind?

contact us and receive a bespoke proposal in 2 business days


Schedule a call


Sunscrapers Sp. z o.o.
ul. Słomińskiego 7/254
00-195 Warsaw

New Business
[email protected]

[email protected]


Clutch award top Python & Django Developers 2019 Clutch award top financial services software developers 2019 Clutch award top B2B companies Poland 2019 Clutch award top developers Poland 2017


Sunscrapers Sp. z o.o.
ul. Słomińskiego 7/254
00-195 Warsaw
New Business
[email protected]

[email protected]

Hi there, we use cookies to provide you with an amazing experience on our site. If you continue without changing the settings, we'll assume that you're happy to receive all cookies on the Sunscrapers website. You can change your cookie settings at any time.

Learn more