Why I Used Kafka for Sending Emails

Many years ago (2014), I had a simple requirement in one of the systems I was working on:

Every time new data was inserted, the system needed to send an email to the recipient.

Sounds simple, right?

For a small number of transactions, it probably is.

The application inserts the data, prepares the email, sends it to the recipient, and continues with the next process.

But then comes another question:

What happens if thousands of records are inserted?

This was one of the reasons I started using Apache Kafka.

The Problem

Imagine an application receiving thousands of transactions.

For every successful insert, an email notification needs to be sent.

A simple implementation might look like this:

The problem is that sending an email is not necessarily fast.

The mail server could be slow. The network could have a problem. The SMTP server could temporarily be unavailable.

If the application waits for the email to be sent every time a transaction is processed, email delivery can start affecting the performance of the main application.

And in my case, email was only a notification.

It shouldn’t slow down the actual business transaction.

Enter Kafka

Instead of asking the main application to send the email directly, I separated the process.

The application only needed to say:

“Hey, this transaction happened and an email needs to be sent.”

Then Kafka could take care of passing that message to another service responsible for sending the email.

The architecture became something like this:

This is where two important Kafka concepts come in:

Producer

The producer sends messages to Kafka.

In my case, after the application successfully inserted the data, it produced a message containing the information required for the email notification.

For example:

{
    recipient: "customer@example.com",
    transactionId: "123456",
    notificationType: "TRANSACTION_SUCCESS"
}

The producer doesn’t need to wait for the actual email to be delivered.

Its job is simply to publish the message.

Consumer

On the other side, we have the consumer.

The consumer listens for messages from Kafka.

When a new message arrives, the consumer reads it and performs the required action.

In this case:

send the email.

So even if thousands of transactions happen, the main application doesn’t need to send thousands of emails directly.

Kafka acts as the layer between the transaction processing and the email processing.

Why Was This Useful?

The biggest advantage for me was decoupling.

The main application was responsible for processing the business transaction.

The email service was responsible for sending emails.

Kafka connected the two.

If email delivery became slow, it didn’t necessarily mean the main transaction needed to become slow as well.

It also made the architecture easier to scale.

If the number of messages increased significantly, additional consumers could process messages from the Kafka topic.

Conceptually:

Instead of making the main application do everything, we distributed the work.

Kafka Is More Than Just Email

Of course, Kafka was not created specifically for sending emails.

Email notification was simply one practical use case where Kafka made sense for me.

The same producer-consumer concept can be used for many things:

• Transaction notifications
• Audit events
• Log processing
• Data synchronization
• Fraud detection
• Analytics pipelines
• Microservice communication

The basic idea remains similar:

Something happens → produce an event → Kafka receives it → one or more consumers process it.

Final Thought

When I first used Kafka, the thing that made the most sense to me wasn’t the terminology around distributed streaming platforms.

It was a much simpler idea:

Don’t make the main application wait for work that can be processed asynchronously.

In my case, thousands of database inserts could potentially generate thousands of emails.

Instead of tightly coupling those two processes, Kafka gave us a buffer between them.

The application could focus on processing transactions.

The consumer could focus on sending emails.

And Kafka sat nicely in the middle.

Sometimes, understanding a technology becomes much easier when we stop asking:

“What does this technology do?”

and start asking:

“What problem did I actually need it to solve?”

If you’d like to explore more of my notes, feel free to check out my other sections on code, databases, and even management – all still connected to the broader world of technology.

That’s all for now, folks! Keep learning, keep building, and most importantly enjoy the journey! 🥰😍