All You Need to Know About Crawlers - Noemi Ferrera, Amazon

All You Need to Know About Crawlers - Noemi Ferrera, Amazon

In this talk, Noemi will cover key concepts about crawlers - what they are, why and when we need them, types of crawlers, and their components. We will see a demo for creating and using crawlers with Selenium, and the pros and cons of doing so.
Outline/Structure

1. What is a crawler?

2. Why and when do we need one?

Discovery testing
Finding particular common issues
Generally in production or pre-prod (late)
Quick coverage

3. Types of crawlers:

API vs. UI
View-only vx. all across / tree crawler
Exhaustive vs. smart

4. Components of a crawler / key concepts:

View / node: how to tell you are in a different one
Arcs: how to navigate / not only clicks, staging, moving objects, not Dom...
Visited storage: not always needed
Heat map: by usage / by issues found / by novelty

5. Crawling with Selenium - examples
Learning Outcome

At the end of this presentation, attendees will have:

Learned key concepts about crawlers
Learned about different types of crawlers
Gained an understanding of when to use each type of crawler
Learned how to identify views and navigating in all types of apps.
Watched how to create crawlers with Selenium.

Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should) also be automated as well. For more infomation check out: https://www.selenium.dev/

Видео All You Need to Know About Crawlers - Noemi Ferrera, Amazon канала Selenium Conference