42nd eBay Tech talk in Berlin: I. Mazor - Calculating User Profiles in Real Time (July 27th, 2017)

42nd #ebaytechtalk: Patterns for Calculating User Profiles in Real Time Igor Mazor

It’s important to know your users’ preferences and behaviour in the E-marketplace world. If you can quickly understand who your users are, you can optimize the user journey on the E-marketplace platform by presenting relevant products to the user, and by improving the relevance of search results. One way to leverage user preferences is to calculate a passive user profile based on the user’s interactions with the E-marketplace platform. At mobile.de, Germany's largest online vehicle marketplace, we include in the user profile information such as likelihood of a user to select different car colors, price distribution, mileage distribution, etc.
The real challenge is designing a scalable system that can calculate profiles for different users in real-time and serve those profiles via a REST API to other stakeholders.

At mobile.de, we reviewed some of the most popular open-source stream processing solutions to consider possible architecture designs for the problem. In a nutshell updating a user profile in real time is actually a stateful stream processing system in which the state is the user profile, the state key could be the user ID, and the state update operation can be as simple as counter increment or an average/variance update.

The main focus of this talk will be: how to store the stream state? Which is one of the most important choices to be made before designing a stateful stream processing system.

During this talk I will compare a local storage option and an external (global) storage option, and will try to present the different trade-offs as a result from selecting each of the mentioned options above.
Igor is a Senior Data Engineer at mobile.de, Germany's largest online vehicle marketplace. Since joining the company in 2016, he has been pioneering the implementation of scalable micro-services around data products. Igor’s current focus is on designing and building a real-time infrastructure that can support millions of users per day.

Before joining mobile.de, Igor has been a BI developer and a data engineer at sparks42 and Rocket Internet, working on scalable DWH infrastructure and data ingestion pipelines.

Igor enjoys solving hard data problems, especially problems which require creative solutions, and he gets satisfaction from seeing his solutions working in a scalable manner, providing the desired results, and helping to drive company business goals.
This talk was organized and hosted by the eBay Tech teams in Berlin. We are software developers, testers, data engineers & scientists as well as site operations teams from eBay Germany, eBay Kleinanzeigen, mobile.de and MOTOR-TALK.

We build leading online platforms using the latest technologies and methods.

Meet us in person at our next Tech Talk in Berlin: https://www.meetup.com/eBay-Europe-Te... .

Visit ebaycareers.com for a list of our open positions.

This video was produced by zaunfilm.de

Видео 42nd eBay Tech talk in Berlin: I. Mazor - Calculating User Profiles in Real Time (July 27th, 2017) канала eBay Tech Berlin