megacolorboy

Abdush Shakoor's Weblog

Writings, experiments & ideas.

Download a webpage along with it's assets using wget

I wanted to download a webpage along with it's assets and scripts, I tried using Chrome's Save as... option but it wasn't working as expected.

I did a little googling and thought of using wget to do the job and it worked. Here's the command I used:

wget -p -k -H --nd https://www.example.com

Let's see what do those flags mean:

  • -p: Downloads all the necessary files to properly display the downloaded HTML page.
  • -k: After the download is complete, it converts the links in the document to make them suitable for local viewing.
  • -H: This would download files that spans different hosts.
  • --nd: While retrieving files recursively, it will not create a hierarchy of directories and downloads the files into a single directory instead.

If you open index.html directly on the browser, the assets might be broken as the -k flag doesn't seem to make the assets relative to the root directory. In order to view it, you can view it on a localhost server like WAMP, XAMPP or if you are using Python, you can type python -m http.server and view your downloaded file.

Might be a bit messy but gets the job done.

Extract private key and certificate from .pfx file

Ever wondered what's a .pfx file? It's just a combination of both private key and the certificate and this file is usually used in Microsoft IIS Servers.

However, you can't use this file in Linux servers as both private key and the certificate are supposed to be individual files.

Here's how you can extract both private key and certificate files:

openssl pkcs12 -in file.pfx -nocerts -out privatekey.pem -nodes
openssl pkcs12 -in file.pfx -nokeys -out certificate.crt

Hope this tip helps you out!

Install NGINX and PHP on Windows

Thought of writing a small tutorial on how to install NGINX and PHP on Windows. You can also use this as a reference if you wanted to install them on Windows Server too.

Note

This tutorial won't teach you how to configure both PHP and NGINX configurations as there are a lot of tutorials focusing on that topic.

Setting up NGINX

  1. Download NGINX for Windows
  2. Go to C:/ directory and create a folder named C:/nginx.
  3. Unzip the downloaded .zip file in C:/nginx directory.
  4. Go to C:/nginx and create two new folders named sites-available and sites-enabled.
  5. To test, if it's working, double-click on C:/nginx/nginx.exe and type localhost in the browser. You should be able to see a Welcome page.
  6. Kill the NGINX process from Task Manager.

Setting up PHP

  1. Download PHP 8.1 for Windows.
  2. Go to C:/ directory and create a folder named C:/php or if you wanted to have multiple versions, C:/php8.1.
  3. Unzip the downloaded .zip file in C:/php directory.
  4. Add PHP to your system environment variables by adding the path C:/php
  5. Open Command Prompt and type php -v. You see should be able to see the version if it's installed correctly.

Run NGINX and PHP as separate services

Generally, it's good practice to run both PHP and NGINX as background services or else, you might have to start the processes manually every time the server goes down.

As I was looking for a way to do that, I stumbled upon this post on StackOverflow and it was quite helpful.

Install NSSM Service Manager

NSSM Service Manager is a free open-source alternative that helps you create a service with the help of GUI. Below are the steps that I followed:

  1. Download NSSM Service Manager
  2. Go to C:/ directory and create a folder named C:/nssm.
  3. Unzip the downloaded .zip file in C:/nssm directory.

Install NGINX as a service

  1. Open Command Prompt as an Administrator.
  2. Go to C:/nssm/win64/ directory.
  3. Type nssm install nginx
  4. Define the path of the nginx.exe file.
  5. Click on Install Service.
  6. Press Win+R and type services.msc.
  7. Look for nginx and start the service.
  8. Open your browser and type localhost to see if it's working correctly.

Install PHP as a service

  1. Open Command Prompt as an Administrator.
  2. Go to C:/nssm/win64/ directory.
  3. Type nssm install php
  4. Define the path of the php-cgi.exe file.
  5. Add the arguments: -b 127.0.0.1:9000
  6. Click on Install Service.
  7. Press Win+R and type services.msc.
  8. Look for php and start the service.

Test to see if it works!

Before you proceed with the following steps, ensure that you include the *.conf files from the sites-enabled folder in your nginx.conf file.

Add the line include "C:/nginx/sites-enabled/*.conf"; in your nginx.conf file and follow the steps below:

  1. Go to C:/nginx/sites-available directory and create example.com.conf.
  2. Go to C:/nginx/html and create directory.
  3. Refer to the sample configuration provided and make the necessary changes.
  4. To enable the site, you have to create a symlink: mklink "C:\nginx\sites-available\example.com.conf" "C:\nginx\sites-enabled\example.com.conf".
  5. Ensure the configuration doesn't have any errors by typing nginx -t in C:/nginx directory.
  6. Restart the NSSM service.
  7. Add an entry in your hosts file: 127.0.0.1 example.com.
  8. And you're done!

Tip

If you get a "Connection Refused" or error that is similar to that, it's most probably due to firewall or maybe the PHP service isn't running.

Useful commands while using NSSM

# Start a service
nssm start "servicename" 

# Restart a service
nssm restart "servicename" 

# Stop a service
nssm stop "servicename" 

# Install a service
nssm install "servicename" 

# Remove/Uninstall a service
nssm remove "servicename" 

Sample NGINX Virtual Host Configuration

server {

    # Your Domain Name
    listen 80;
    server_name example.com;

    autoindex off;
    add_header X-Frame-Options "SAMEORIGIN";
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Content-Type-Options "nosniff";
    add_header X-XSS-Protection "1; mode=block";
    add_header Access-Control-Allow-Origin *;

    if (!-e $request_filename) {
        rewrite  ^/[_0-9a-zA-Z-]+(/wp-(content|admin|includes).*) $1 break;
        rewrite  ^/[_0-9a-zA-Z-]+(/.*\.php)$ $1 break;
    }

    # Uncomment these lines if SSL is provided
    #==========================
    #listen 443 ssl;
    #ssl_certificate your-ssl.pem;
    #ssl_certificate_key your-ssl.key;
    #ssl_protocols TLSv1 TLSv1.1 TLSv1.2; 
    #ssl_prefer_server_ciphers on;
    #ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    #==========================

    # Log files for Debugging
    access_log logs/example-access.log;
    error_log logs/example-error.log;

    # Directory
    root "C:/nginx/html/example.com";
    index index.php index.html index.htm;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location ~ /\.(?!well-known).* {
        deny all;
        access_log off;
        log_not_found off;
    }

    # Deny yaml, twig, markdown, ini file access
    location ~* /.+\.(markdown|md|twig|yaml|yml|ini)$ {
        deny all;
        access_log off;
        log_not_found off;
    }

    # PHP-FPM Configuration Nginx
    location ~ \.php$ {
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
        include        fastcgi_params;
    }
}

Conclusion

If you've gotten this far, give yourself a pat in the back.

Hope you've found this tutorial useful and please share it with your friends and colleagues if they really need it!

Count the number of matches of a pattern in VIM

You can the count the number of matches of a pattern by using the n flag while using the substitute command. Try the following command:

:%s/something//gn

If you want to know on how many lines it matches, just omit the g flag:

:%s/something//n

Hope this tip helped you out!

Count the number of word occurences in a text file using grep

If you are using a GUI based application like Microsoft Word of Google Docs, it would be easier for you to know the number of word occurences in a file. But what if you are in a terminal? That's where both grep and wc comes handy tools.

Let's say you have the following text:

The European languages are members of the same family. Their separate existence is a myth. For science, music, sport, etc, Europe uses the same vocabulary.

Now, you can use the grep and wc tool to count the number of times "the" appears in the file:

grep -o -i "the" file.txt | wc -l

Hope this helps you out!

Install MariaDB 10.3 on Void Linux

I tried to find a proper guide on how to install MySQL 8.0 on Void Linux but I couldn't really find any. So, I thought of installing MariaDB 10.3 on my laptop instead.

Don't worry about using MariaDB as it meets the same standard enterprised requirements as MySQL. The only difference is that MySQL belongs to Oracle and MariaDB is for people who wanted to get out of Oracle's hands.

Anyway, let's get started.

Prerequisites

  • Need to have root privileges in order to install packages.
  • Basic know-how of Void Linux and it's xbps package manager.

Alright, go ahead and follow these steps one-by-one:

1. Install MariaDB

Type the following command to install the package:

sudo xbps-install -S mariadb

2. Initialize MySQL data directory

The following command will initialize a MariaDB data directory and create system tables in the MySQL database, if they are not present.

mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql

The options used:

  • --user: The login user name to use for running the mysqld process.
  • --basedir The path to the MariaDB installation directory.
  • --datadir: The path to the MariaDB directory.

To know more about it, you can refer to MariaDB's documentation.

3. Enable MySQL service

If you are used to Ubuntu or CentOS, enabling services in Void Linux is a bit different but quite simple. Once a process is symlinked, it will start on boot and restarts if it stops unless you stop the service deliberately.

Type the following command to enable the mysqld service:

sudo ln -s /etc/sv/mysqld /var/service/mysqld

4. Start MySQL service

Type the following command to start the mysqld service:

sudo sv start mysqld

To confirm, if the service is running, type the following:

sudo sv status mysqld

And you'll see something like this:

run: mysqld: (pid 15136) 1116s; run: log: (pid 15025) 1172s

5. Secure MySQL installation

It's recommended to secure your installation, so type the following command:

sudo mysql_secure_installation

Once the command is executed, you'll be prompted with a few questions, respond according to your needs and you're done!

6. Test MariaDB connection

Now, all you have to do is, try to log in to your database by typing the following:

mysql -u root -p

Once executed, it would prompt you for your password (which you must have set in the previous step and if you are able to log in, you are good to go!

Hope this guide helps you out!

How to create an ISO image from CDs/DVDs?

I used to create ISO images using DAEMON Tools but I wanted to try something different and see if there's a way to create it using the Linux CLI.

Turns out, you can do in just a single line using the dd utility like so:

dd if=/dev/cdrom of=/path/to/your/directory/image.iso

The if stands for input file and of stands for output file.

Looks like, I don't have to use DAEMON Tools for stuff like this, I guess.

Hope you found this tip useful!

Truncate a file using redirection in Linux

Simply put, sometimes, there are situations in where you just want to clear the contents of a file without deleting it.

This could be for many reasons like to avoid permission related issues, or maybe the file could be having useless logs that amasses to a size that measures in GBs.

So, the easiest solution is to clear it away from a terminal is by shell redirection like so:

:> filename

Let me break down the command here:

  • The : symbol means true and doesn't produce any output.
  • The '>' symbol is used for redirecting the output of the preceding command (in this case, it's empty!)
  • filename is the file that you want to truncate. If it doesn't exist, the file will be created.

Alternatively, you can do the same by using the cat command to output the contents of the /dev/null device (which only contains a EOL character) to empty a file:

cat /dev/null > filename

Hope this comes in handy!