Building an Amazon Price Monitor using Bash Scripting

Bash gets a lot of flak from everyone. It’s quirky, has too many ways of doing one thing, and its syntax is atrocious. I belonged to the same camp and avoided bash scripting with a ten-foot pole, but that changed recently. I had to deal with bash in my full-time job, and saw that for all its cons, bash gets the job done faster than anything else. The breeze with which you can combine tools makes it a worthwhile tool for most scripting needs.

In this tutorial, we are going to build a simple product price monitor in bash. We are trying to send a notification whenever the product price changes.

Use shellcheck

A super-useful tool to deal with bash’s quirkiness is shellcheck. It’s a linting tool for shell scripts and proved to me to be an invaluable companion in avoiding the pitfalls. It has extension for most of the code editors, so do install it before going ahead.

Fetching Price and Product name

In bash, variables are declared using this syntax VAR_NAME="Hello world". And the same syntax can be used to declare multiline strings. So, we are going to first declare a variable to store list of product links that we are going to monitor.

 
PRODUCTS="
https://www.amazon.in/dp/B07X4R63DF/
https://www.amazon.in/dp/B07HGJJ58K/
https://www.amazon.in/dp/B07J3CJM4N/
" 

There are three ways to loop in bash. The one we are going to use is for .. in loop.

 
for PRODUCT in $PRODUCTS; do
    echo "$PRODUCT";
done 

What’s happening here? In bash, the for ... in loop iterates over series of strings separated by IFS (Input Field Separator), whose default value if “<space><tab><newline>“. That means, when iterating over $PRODUCTS each newlines represents one iteration. We would have had problems if the string had spaces or tabs, but we are okay since it’s not there.

Also, you can see that I used $ in some case and not in others. That’s just how bash works. The general rule is that you add a $ when you’re reading a variable and omit when you’re setting it.

Let’s fetch the product information from Amazon:

 
for PRODUCT in $PRODUCTS; do
    PAGE_CONTENT=$( curl -sL -H "User-Agent: Chrome" "$PRODUCT" )

    PRICE=$( echo "$PAGE_CONTENT" |
        pup "#priceblock_ourprice text{}" |
        perl -nle 'print $1 if /([0-9,]+)/' |
        sed s/,//g
    )
    PRODUCT_NAME=$( echo "$PAGE_CONTENT" | pup "#productTitle text{}" | xargs )

    echo "$PRICE" "$PRODUCT_NAME"
done 

So, what’s going on here? This might look scary bit, but with a a bit of experience you’re able to quickly understand the bash way of doing things. A few things, There’s no one way of doing things in bash. You can combine other tools or use a completely different approach. This has its side-effects, but I find it boon to my productivity as it allows me to build things really fast.

Storing and Comparing

Now that, we have fetched the product price and the name, it’s time to store it somwhere. Again, it’s upto you what you use - redis, db, or plain text files. My favorite solution is SQLite. You can create a small DB, and store and query easily.

 
#!/bin/bash

PRODUCTS="
https://www.amazon.in/dp/B07X4R63DF/
https://www.amazon.in/dp/B07HGJJ58K/
https://www.amazon.in/dp/B07J3CJM4N/
"

touch "$HOME/.amazon.db"
sqlite3 "$HOME/.amazon.db" "
    CREATE TABLE IF NOT EXISTS products (
				url text PRIMARY KEY,
				price integer,
				name text
		)
"

for PRODUCT in $PRODUCTS; do
    PAGE_CONTENT=$( curl -sL -H "User-Agent: Chrome" "$PRODUCT" )
    PRICE=$( echo "$PAGE_CONTENT" |
        pup "#priceblock_ourprice text{}" |
        perl -nle 'print $1 if /([0-9,]+)/' |
        sed s/,//g
    )
    PRODUCT_NAME=$( echo "$PAGE_CONTENT" | pup "#productTitle text{}" | xargs )

    OLD_PRICE=$( sqlite3 "$HOME/.amazon.db" "
			SELECT price FROM products WHERE url='$PRODUCT'
		" )

    if [[ -n "$OLD_PRICE" && "$OLD_PRICE" -ne "$PRICE" ]]; then
        osascript -e "display notification '$PRODUCT_NAME's price has changed to $PRICE'"
    fi

    sqlite3 "$HOME/.amazon.db" "
        INSERT INTO products
        VALUES('$PRODUCT', '$PRICE', '$PRODUCT_NAME')
        ON CONFLICT(url) DO UPDATE SET price = EXCLUDED.price
    "
done 

Let’s understand new things added line by line:

1. touch is a useful command to create a file that doesn’t exist. Although the real purpose of the command is to change file access time.
2. We, then, create a products table. IF NOT EXISTS makes sure that SQLite doesn’t try to re-create the table.
3. After fetch product data using cURL, we try to fetch the price for the product in the table. If it exists OLD_PRICE will be assigned that value. If not, it’ll be empty.
4. In bash we us [[ to create a conditional. First we check if the variable is empty (i.e, the product is new and was fetched for the first time). -n is bash’s flag to check if the string’s length is nonzero. If it’s not empty, we check the if old price and new price are not equal.
5. If yes, we send a popup notification using an Applescript Code. (Note: this will be different for Windows and Linux).

So, our script is ready. products.sh will fetch details of Amazon products, store it in a DB, and send us a notification if the price has changed.

To be more useful, we should run the script at regular intervals using crontab. Run crontab -e and add this line,

 0 * * * *	/path/to/script/products.sh 

Conclusion

I am not sure if I did a good job here convincing of a bash solution. There’s a bit of learning curve here, and it reminds me the sort of curve I had learning git. But once past that, the ability to build stuff goes up dramatically. And it’s not difficult to see why. When working with HTTP requests, curl’s syntax is much more terse and powerful.

To create files, you just have to redirect the command’s output using redirection operator. To use SQLite, you don’t have to learn the right package in your language, you can use the CLI tool. To send a notification, you need only one line of code.

I wouldn’t want you to build a web backend with bash, but you should give it a try for the next automation.

Happy bashing!