A social media monitoring service that needed to make a new highly scalable architecture.
Alerti is a social media monitoring and analysis solution. This application makes possible for a company to collect all the data from different social media platforms to analyze it's brand's performance and to take action.
The main part of this Ruby on Rails application consists on crawling the internet to collect all information requested by the users. Due to its constant growth, this system started to work slowly as it was handling hundreds of millions of rows.
My first mission was to research and select the most fitting relational database that could handle the growing number of users. I analised different technologies, benchmarking for different databases, and optimizing db indexing and structure.
The next task was to refactor and rewrite the new crawler architecture that could process the data faster and more efficiently. I designed the new architecture, and implemented it using Ruby and Sidekiq.
