megacolorboy

Abdush Shakoor's Weblog

Writings, experiments & ideas.

One year of writing TIL articles

A little self-reflection on how writing TIL articles has improved my productivity and learning.

Last year, around this time, I was working from home due to the COVID-19 restrictions placed by the UAE Government.

During this time, I noticed that I ran out of ideas for my blog and didn't even get time to maintain it for a long time.

Then, I came across Josh Branchaud's TIL collection, which inspired me to start my own TIL section in my blog. At first, I had the idea of merging it with my main blog but then, I thought of keeping it separate and well, fast-forward to a year now, it's been one of the most productive platforms for writing short articles based on whatever I have learnt or solved.

As for those of you who might not know, TIL a.k.a stands for Today I Learned and what I really liked about it is that there isn't any barrier that stops you from writing a short article and that was the one thing that motivated me to write something on my blog.

Has it helped it you?

Yes, it did help me in two ways.

Professionally, it has motivated me to learn more about my field and share my knowledge about whatever I learned while solving a particular problem or a tip that I found useful. The idea was likened to a personal StackOverflow repository that I would often revisit to refresh my memory.

Personally, it helped me reflect my values as a programmer and made me realize that there's a lot of things to learn out there and it's pretty much endless as there's always something to learn every single day. And yes, it helped me improve my writing abilities too.

Although, this article resides in a what I would call it as a "Technical blog", I guess, this applies to everyone who wants to refresh their writing productivity and help them get started back on the tracks again.

Hope you liked reading this article.

Stay tuned for more!

Understanding SPF, DKIM and DMARC protocols

An insight into the three main email security protocols that protects your email from malicious attackers.

Today, email spam is one of the most common cyberattacks conducted by people with malicious intent intending to steal your passwords and personal credentials, leading users to phishing sites to steal bank account details, identity theft and so on.

Because of this, ISPs and email providers such as Gmail and Office 365 are taking anti-spam measures by enforcing stricter protocols in the type of emails that they receive, so it's good to implement those protocols and ensure that your email gets delivered and not delayed or worse, rejected by the mail servers.

So, what are those protocols?

SPF, DKIM and DMARC are the three main secure protocols used to authenticate your mail server and this will prove the ISPs and other mail service providers that the mail being sent is legitimate and authorized.

What is SPF?

It's an acronym for Sender Policy Framework. SPF is nothing but a DNS TXT record that specifies the server(s) and IP addresses that are allowed to send email from a specific domain.

Assuming you are the sender, just think of sending a postcard to your friend in which you add your address as well, so that your friend knows who the recipient is and he/she would most likely open it because they trust it.

But in technical terms, the actual recipient is not the user but rather the mail server that receives the mail.

Create an SPF record

An SPF record is a very simple string and it can be easily created. However, there are a few parts to it:

  1. Version of SPF used.
  2. IP addresses that are authorized to send emails.
  3. Third party domain(s) that are authorized to send emails.
  4. An ending tag named "all" which tells the receiving server on what policy to apply if the sending server is not a part of the SPF record.

So, let's what does it look like and what each part of it does:

v=spf1 ip4:111.111.111.111 include:example-domain.com -all

v=spf1 states the SPF version being implemented. Currently, there's no any other version at this point. So, it should always stay as this version until another version is released.

ip4:111.111.111.111 is the IP address of the mail server/domain that's authorized to send emails for that domain. You can use multiple IP addresses and can be listed individually like this ip4:111.111.111.111 ip4:222.222.222.222 or through a CIDR like ip4:111.111.111.0/20. If both IPv4 and IPv6 addresses are being used by mail server, make sure that both of those addresses are being listed.

include:example-domain.com is a secondary domain that's authorized to send emails on behalf of the primary mail domain(s) listed. Just like the previous rule above, you can add multiple secondary domains but bear in mind that only a maximum of 10 domains are allowed for any sending domain.

-all is a tag that instructs the receiving server on how to handle messages from a domain that isn't a part of the SPF record. There are some options and they are all dictated by a single character that precedes the all keyword. The options are:

  • -all means hard fail. The receiving server should reject the email if the sender domain is not authorized.
  • ~all means soft fail. The receiving server can flag it as a possible spam if the sender domain is not authorized.
  • +all means authorized. The receiving server allows the email even if the sender domain is not authorized. Now, this is not recommended and please do not use this option.

What is DKIM?

It's an acronym for DomainKeys Identified Mail a.k.a Email signing. Just like SPF, DKIM is also a simple DNS TXT record that tells the receiving server that the mail is certified which allows to build a trust between both the sending and receiving servers.

To handle this trust, DKIM makes uses of an RSA cryptographic algorithm to create a pair of public and private encryption keys. The private key will remain on the server (i.e. the mail server) whereas the public key is placed in your DNS records.

How to create a DKIM record?

Depending on your mail provider, it can be easily generated by tools provided by them and once generated, it can be copy-pasted to the DNS records.

This is how a typical DKIM record would look like:

v=DKIM1; k=rsa;
p=iHeFQ+7rCiSQs3DPjR2eUSZSv4i/Kp+sipRfVH7BGf+SxcwOkX7X8R1RVObMQsFcbIxnrq7Ba2QCf0YZlL9iqJf32V+baDI8IykuDztuoNUF2Kk0pawZkbSPNHYRtLxV2CTOtc+x4eIeSeYptaiu7g7GupekLZ2DE1ODHhuP4I=

Regardless of how it's being generated, this is what each part of the header means:

v=DKIM1 is the DKIM protocol version used.

p= is the Base64 encoded public key generated.

k= is the mechanism used to decode the DKIM signature. The encryption key is usually based on rsa-sha1 or rsa-sha256 signing algorithm.

You can use DMARC analyzer's DKIM checker to check if your DKIM record is valid.

What is DMARC?

It's an acronym for Domain-based Message Authentication, Reporting and Conformance. This protocol is built around SPF and DKIM and it ensures the following:

  1. Verifies that the sender's email is protected by SPF and DKIM protocols.
  2. Instructs the receiving mail server on what to do if the authentication fails.
  3. Provides a way for the receiving server to send a report to the sender about the DMARC evaluation i.e. whether it passed or failed.

If you have come this far in the article, you might have understood why both SPF and DKIM were explained and why they are quite necessary.

How to create a DMARC record?

Once you have both SPF and DKIM records in place, then it's easy to create your DMARC record. There are many sites and I would recommend you to try MXToolbox's DMARC Record Generator.

This is how a DMARC record would look like:

v=DMARC1; p=none; fo=1; rua=mailto:address@example.com;

And here's what each part of the header translates to:

v is the version tag, similar to the SPF record. It should always be DMARC1 in the record.

p is the policy tag. none means to not do anything to the email, quarantine means to flag it as spam and reject means to reject the email.

fo is the tag that lets the receiving servers know that failed messages must be returned to the sender or not. There are four values for this tag:

  • 0: Generate a report if both DKIM and SPF produce a "Pass" result.
  • 1: Generate a report if both DKIM and SPF produce a result other than "Pass".
  • d: Generate a report if the email had failed the DKIM evaluation.
  • s: Generate a report if the email had failed the SPF evaluation.

It's actually recommended to use fo=1 which can help you look for any email delivery issues.

rua tells the receiving server on where to send the aggregate reports. This could provide insights into the health of the email server and can help identify any malicious activities.

There are many optional tags that can be used but these are the tags are most commonly used by default.

Conclusion

With the increase in spam emails, it's good to take such preventive measures to ensure that you are following best practices and doing your part to prevent malicious emails and other security related issues.

If you want to read more about these protocols, you can read the following links recommended below:

Hope you liked reading this article! 😄

Find directories created within a date range

Executing these commands helps me create a sorted list of files/directories created within a specific date range:

touch -t 202104100000 start
touch -t 202104150000 stop
find . -type d -maxdepth 1 -newer start \! -newer stop | sort >> directories.txt

Someday, these commands shall come in handy, bud! 😎

Perform Git operations using path directory

The -C flag means the path of the directory and using this flag, you can perform any Git operations outside the project's directory without having to enter the directory all the time:

git -C /path/to/directory <command>

Hope this tip helps you out!

Exclude directories while searching for a pattern in files

Looking for a specific text pattern in a directory but wanted to avoid some paths? Here's a quick command that you can try:

grep -R --exclude-dir=path/to/directory 'some pattern' /path/to/search

Hope this helps you out!

Fetch selected files from your remote repository

Wanted to fetch a specific file from your Git repository except that the repository doesn't exist in your local machine?

Try this out:

git init
git remote add origin <your_repo_link>.git
git fetch
git checkout <your_branch_name> -- </path/to/file>

After executing these commands, you should be able to see the selected directory/file in your project directory.

Hope you found this useful!

Find directories older than a specific date and sorted by size

Wanted to see which directories were created on an older date along with their sizes? Try this:

find . -type d -maxdepth 1 ! -newermt "2021-04-10" -exec du -sh {} \; | sort -h >> oldprojectsizes.txt

Hope you found this tip useful!

How to change the character encoding of a file via Terminal?

Sometimes, I face character encoding issues while making minor edits via a SFTP console connected to a Linux server. I found a quick hack to change the file encoding using vim on the command line.

In this example, I'm changing the encoding of the file to unix:

vim $filename +"set ff=unix" +wq

Hope you found this helpful!