Publish-Subscribe Architecture (Explained by Example)

In this video, I want to discuss the Pub sub architecture or publish subsribe architecture and talk about the pros and cons and differences over Request Response.
One of you guys asked me which is better REST or Message queue. I struggled to answer for a while but After thinking about it I discovered that both REST and Message queue does boil down to basic elements. REST being HTTP which is Request-Response and Message Queue being Publish subscribe.

Timecodes
0:00 Intro
3:00 Where does request response pattern breaks
7:00 Response request pros & cons
11:12 Pub/sub pattern
17:50 Pub/sub pros/cons
- [ ] Request/ Response
- [ ] Pros and Cons
- [ ] Publish-Subscribe Definition
- [ ] Pros and Cons

Req/res

Browsing
Fetch or XMLHttpRequest
SQL query

Pros
- Simple to implement (REST)
- Stateless (HTTP)
- Sync or Async (service and client)

Uploading a youtube video
- Compress video
- format video
- Notify subscribers

Upload breaks down the video into small chunks and uploads each one

Once all successfully uploaded send a whole video to the compress service

One video compressed, the compress service sends to the format service and notifications service

If you are only using pure request-response, The client will be waiting until all of that stuff is done and if the connection is broken or any failure happens you lose.

Plus let's assume You want to also add a feature where the video will need to be checked for copyrighted material.. so the compress service now need to make a request to the copyright service and make sure all of that requests return successfully

Cons
- Doesn’t scale with multiple receivers
- Creates extreme coupling between multiple services
- The client and servers have to be running to communicate.
- chaining, retries, circuit breaking
- Chain effect and circuit breaking (microservices)
Meet the publish-subscribe model

Ok, this is starting to sound like a commercial.

Publish subscribe or pubsub for short is an architecture model where a publisher posts. a message to a topic (usually in the form of a queue) and clients optionally can subscribe to the topic and get notified when a new message is posted.

Upload breaks down the video into small chunks and uploads each one

Once all successfully uploaded send a whole video to message queue or topic

The compress service subscribes to the topic. The new video will get received by the compress service, compresses the video and publish it back to the topic or queue as a compressed.

The notification and copyright service subscribes to the topic.

Redis/kafka/RabbitMQ

Pub sub
Pros
- Scales with many receivers
- Loose coupling(publishers don’t know the subs, subs don't know the pubs)
- Caching
- Work while publishers or subscribers are offline
- Great with microservices avoid chaining, retries, circuit breaking
- Pairs with event-driven architecture
Is the pub/sub perfect though?

Decoupling side effect is a serious problem. You just decoupled the sub from the publisher, how do you guarantee to deliver a message?

Cons
- Message delivery issues (exactly once delivery ar least once) https://youtu.be/IP-rGJKSZ3s
- Network saturation- (depends on Pubsubs system) in push model clients might get overwhelmed, in pull model false attempts
- scaling is still an issue
🐘 Postgres Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQWGrOqslniFlRcwxyY94cjj

🧮 Programming Pattern Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQV1emqxKbcP5esAf4zpqWpe

🛡 Web Security Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQU3YDMRSMvzflh_qXoz9zfv

🦠 HTTP Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQU6zO0drAYHFtkkyfNJw1IO

🏰 Load Balancing and Proxies Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQVMeBmWI2AhxULWEeo7AaMC

🏭 Software Architecture Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXNP6mQchJVP3S-3oKGEuw9

🛰 Network Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQUBSgBXilKhRMJ1ACqr7pTr

💾 Database Engineering Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQXjD0HOzN7P2tgzu7scWpl2

🐍 Python Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQU_M83ARz8mDdr4LThzkBKX

🔆 Javascript Videos
https://www.youtube.com/playlist?list=PLQnljOFTspQWab0g3W6ZaDM6_Buh20EWM

Support me on PayPal
https://bit.ly/33ENps4
Stay Awesome,
Hussein

Видео Publish-Subscribe Architecture (Explained by Example) канала Hussein Nasser