April 25

Variables, Pointers, and Indirection in Arduino C

Before we continue on with learning about the I2C protocol and our EEPROM project, we need to discuss variables: what they are and what goes on behind the scenes. Knowledge of how variables work and the use of pointers and indirection with arrays will serve us well when it comes time to read from our EEPROM. Let’s begin.

Anatomy of a Variable:

1. What is a variable?

Simply put, variables hold data. More specifically, a variable holds data of a specific data type. For example, an int holds an integer, a string contains a collection of chars, etc.

2. What goes on behind the scenes when a variable is defined and when it is assigned?

When you define a variable, the compiler goes and checks the symbol table (basically a list of variables that have previously been declared) to see if that variable already exists. If it doesn’t, the compiler goes ahead and adds the new variable to the list.

Say, for example, you add the following statement:

int myVar;

Since our variable has not already been declared (it doesn’t already exist in the table), the compiler updates the symbol table so it now looks like this:

Symbol table with myVar declared (but not yet defined) since it lacks a location in memory (lvalue).
Symbol table after myVar declared- note the lack of an lvalue. This is because myVar is not yet defined. rvalue is also unknown because we haven’t assigned a value to myVar yet.

Now, technically, the variable has only been declared at this point- it’s missing an actual location in memory. To get this location in memory, the compiler requests a place to put this variable from the system’s memory manager. The memory manager then responds with a memory address which the compiler then adds to the symbol table for that variable. This memory address is known as an lvalue (lvalue = location value) and it merely represents where the variable can be found in memory. With this addition of the lvalue to the symbol table, our variable is now defined:

myVar now defined in the symbol table (myVar now has an lvalue).
Symbol table with myVar defined- this means that the variable now has a location in memory (lvalue).

With our new variable defined, we can now move on to storing a value in it. Fortunately, assigning a value to a variable is rather straightforward. When we assign a value to a variable, we directly navigate to the variable’s location in memory (the lvalue) and update the memory at that address with the new value. The data that’s actually stored in memory is known as the rvalue (rvalue = register value).

Continuing our example with the following assignment statement:

myVar = 10;

With this assignment, our symbol table now looks like this:

myVar after rvalue assignment
Symbol table after assignment- note the updated rvalue which holds our data value.

Another way to visualize what we have just gone over is with an lvalue-rvalue diagram:

lvalue-rvalue diagram for a value type variable
lvalue-rvalue diagram

This diagram is why you will see some people refer to the memory address as the “left value” and the actual data value as the “right value”.

  • There’s also an important caveat here: in Arduino, and C in general, there is no duty to clear that rvalue at our variable’s lvalue when we define it. Therefore you should always assume that a variable’s value contains whatever garbage was originally in that memory location unless we’ve explicitly assigned a value to the variable. (i.e., Don’t assume it’s 0 or null). Therefore it’s probably best to go ahead and initialize your variable with a value when you define it.
    Let’s summarize: Whenever your program needs to use the value stored in a variable, it uses the variable’s lvalue to go to that memory address and retrieves the data (rvalue) from that memory location.

Pointers:

Now that we’ve covered what variables are and how they really work, we’re ready to understand pointers. Simply put, a pointer is nothing more than a variable that references the memory address of another variable. Using the terminology that we’ve just learned, a pointer is a variable whose rvalue is the lvalue of another variable.

To visualize this, let’s take a look at two lvalue-rvalue diagrams representing the value type variable myVar and the reference type variable myPointer:

myPointer referencing myVar - Notice how the rvalue of myPointer is the memory address of myVar.
myPointer referencing myVar – Notice how the rvalue of myPointer is the memory address of myVar.

Declaring a Pointer:

Declaring a pointer variable is rather straightforward:

int *myPointer;

The type specifier (int in this case) must match the data type of the variable the pointer is to be used with. The asterisk indicates to the compiler that myPointer is a pointer. Since whitespace doesn’t really matter in C, the asterisk can be placed anywhere between the type specifier and the pointer variable name so you will sometimes also see: int* myPointer, int * myPointer, etc.

The Address-Of Operator:

By itself, a pointer that is defined but does not actually point to anything is a pretty pointless pointer (ha!). To point it to the memory address of another variable we simply need to assign the pointer the memory address of that variable. But where do we get the memory address from? That is, where do we get the lvalue of myVar from? Enter the address-of operator (&).

The address-of operator is a unary operator that returns the lvalue of a variable.

Pointer Assignment:

To point our new pointer at the memory location of our value type variable, myVar, we simply call the following statement:

myPointer = &myVar;

This completes the link shown in the previous diagram and is known as referencing. It is for this same reason that the address-of operator (&) is also known as the “referencing operator“.

Whenever you are learning a new concept, it’s a good idea to try it out yourself to prove to yourself what you’ve read. Let’s mock up an example of what we’ve learned so far in the Arduino IDE:

void setup() {
  Serial.begin(9600);
  
  int myVar = 10;  // Initialize a variable.
  
  Serial.print("myVar's lvalue: ");
  Serial.println((long) &myVar, DEC);  // Grab myVar's lvalue
  Serial.print("myVar's rvalue: ");
  Serial.println(myVar, DEC);
  Serial.println();
  
  int *myPointer;   // Declare your pointer.
  myPointer = &myVar; //Assign myVar's memory address to pointer.
  
  Serial.print("myPointer's lvalue: ");
  Serial.println((long) &myPointer, DEC);  //myPointer's lvalue
  Serial.print("myPointer's rvalue: ");
  Serial.println((long) myPointer, DEC);  //myPointer's rvalue
}

void loop() {
}

Watching the serial monitor, what you should see is something like this:

Serial log showing that the rvalue of a pointer is the memory address of the value type variable it references.
Note that the rvalue of myPointer is the same as myVar’s lvalue.

Notice that myPointer’s rvalue is the memory address of myVar (i.e. myVar’s lvalue), just like it shows in the diagram.

Indirection (Dereferencing):

We just saw that a pointer can reference a location in memory by assigning that pointer a variable’s memory address using the reference operator (&). We can take this a step further and obtain the actual value stored at that memory address by dereferencing the pointer. This is also known as indirection and is accomplished via the indirection operator (*) with your pointer. Example:

*myPointer = 5; // Go to memory addressed stored in myPointer's rvalue (myVar's lvalue) and place the value 5 in that memory address.

Continuing off our previous Arduino code example:

void setup() {
  Serial.begin(9600);
  
  int myVar = 10;
  
  Serial.print("myVar's lvalue: ");
  Serial.println((long) &myVar, DEC);
  Serial.print("myVar's rvalue: ");
  Serial.println(myVar, DEC);
  Serial.println();
  
  int *myPointer;
  myPointer = &myVar;
  
  Serial.print("myPointer's lvalue: ");
  Serial.println((long) &myPointer, DEC);
  Serial.print("myPointer's rvalue: ");
  Serial.println((long) myPointer, DEC);
  Serial.println();

  *myPointer = 5;  //THIS IS OUR DEREFRENCING ADDITION.
  Serial.println("-----------------------");
  Serial.println("Updating *myPointer = 5");
  Serial.println();

  Serial.print("myPointer's lvalue: ");
  Serial.println((long) &myPointer, DEC);
  Serial.print("myPointer's rvalue: ");
  Serial.println((long) myPointer, DEC);
  Serial.println();

  Serial.print("myVar's lvalue: ");
  Serial.println((long) &myVar, DEC);
  Serial.print("myVar's rvalue: ");
  Serial.println(myVar, DEC);
  Serial.println();

}

void loop() {
}
dereferencing the pointer and assigning a value; we are able to manipulate the data stored in myVar
Notice that by dereferencing the pointer and assigning a value, we are able to manipulate the data stored in myVar.

Notice that nothing changed to myPointer at all (blue). Neither its lvalue nor its rvalue changed. Contrast that with myVar (red) which had it’s rvalue changed to 5 by the indirection operator we applied to our pointer.

That is the power of pointers and indirection. In my next journal entry, I will discuss pointers and arrays which will then allow us to finally move on to the last part of our EEPROM I2C project!

February 2

How To Update/Install FileZilla on Ubuntu

FileZilla is an incredibly useful FTP client for transferring files between your workstation and servers. In this tutorial, I will walk you through updating/installing FileZilla on Ubuntu without using the repository. In general, if you want the latest and greatest features, try to avoid repositories- the apps in repositories are often outdated. I also feel like a repository is a crutch in that it obfuscates how your software is actually installed on your Linux system.

I can already hear the outrage now; I’m not saying that repositories are worthless. They greatly reduce maintenance when it comes to keeping your system (relatively) up-to-date. Downloading, extracting, and compiling every application from source would be hugely impractical. I use repositories for things that either I don’t use very often or that I don’t care about having the latest version of. For apps that I use often, where I care about having the latest, I handle those manually. Now, on to the tutorial.

1. Obtain your update files.

Obtain your update files. If you already have FileZilla installed, FileZilla checks automatically for updates at launch and downloads them to your home downloads folder. If you don’t have FileZilla already, download it here.

2. Navigate to your downloads folder.

Navigate to your Downloads folder and find your FileZilla tar file. I’ll admit I use the Files GUI app that comes with Ubuntu most of the time. Right click and select “Open in Terminal” (or just open a terminal with Ctrl + Alt + T and just type cd ~/Downloads/).

3. Extract your tar file.

Extract your tar file using the following command:

tar -vxjf FileZilla_3.40.0_x86_64-linux-gnu.tar.bz2

This will extract the file to a directory in your Downloads directory called FileZilla3. You should now have the following:

Extracted FileZilla3 files in ~/Downloads/ directory.
Extracted FileZilla3 files in ~/Downloads/ directory.

Notice this extraction contains a bin directory, implying that it’s ready to run (no compilation necessary).

4. Move your extracted files to their final location.

Let’s move this folder to our /opt/ directory with:

sudo mv ./FileZilla3/ /opt/

But wait! If you’ve already, installed, you’ll get the following error:

mv: cannot move './FileZilla3/' to '/opt/FileZilla3': Directory not empty

Even with sudo, mv will refuse to merge a directory. It’s a nice guardrail. In our case though, we do want to merge. For that, we’ll use rsync:

sudo rsync -a ./FileZilla3/ /opt/FileZilla3/

Warning: DO NOT FORGET to add the /FileZilla3/ directory to /opt/ like it shows above. If you simply did /opt/ you’d wipe out your entire /opt/ folder and be left with only FileZilla3. 

And with that, we’re done!

Category: Linux | LEAVE A COMMENT
January 16

Setup an NGINX Reverse Proxy on a Raspberry Pi (or any other Debian OS)

If you’re running a web server out of your homelab (and you should), you really should consider running your servers behind an NGINX reverse proxy. Honestly, this should be the first thing you build in your homelab. It doesn’t take a lot to setup- NGINX is so efficient it can even be run on something as simple as a Raspberry Pi and it pays you back in dividends once you’ve got it up and running.

What Does a Reverse Proxy Do?

A reverse proxy serves as a sort of dispatcher by acting as a central contact point for clients. Based on the information requested by the client, it then routes the request to the appropriate backend server and makes sure the backend server’s response makes it back to the appropriate client.

Prototypical NGINX reverse proxy diagram

What are these dividends you speak of?

A reverse proxy can give you additional flexibility, security, and even a performance bump. It can also greatly simplify your deployment:

  1. Flexibility: An NGINX reverse proxy can allow you to host multiple sites/domains with only one IP address. It accomplishes this by listening on a port (usually port 80 for HTTP traffic) and parsing the http request header for the host. Based on the host specified in the header, NGINX can route a request to the proper backend server (in a reverse proxy, this is also known as an upstream server).
  2. Security: By standing between the client and the backend server, the reverse proxy provides a degree of separation.
  3. Improved performance: NGINX can be used to cache static content which means that not only is content returned faster to the client, but since it often means that the upstream server doesn’t even need to be contacted, it can take a lot of the load off your backend servers.
  4. Simplifies your deployment: If you’re hosting multiple sites, an NGINX reverse proxy can greatly simplify your implementation by giving you a single point to manage your traffic. This means you only have to set up port forwarding once and whenever you create a new site, all you have to do is add an additional configuration listing to NGINX. When you implement HTTPS (and you should), instead of having to implement it on every individual web server you have setup, you can handle it all on your NGINX reverse proxy.

Installing NGINX:

Now that I’ve hopefully convinced you to implement an NGINX reverse proxy, let’s get started. I recommend using a dedicated device for this (again, it need not be expensive, even a Raspberry Pi will do) but it helps keep everything clean and compartmentalized. With a clean install of Ubuntu (if using an x86-64/AMD64 device) or Raspbian (if on the RPi), do the following:

1. Update your package list and make sure your device is updated:

sudo apt-get update
sudo apt-get upgrade

2. Depending on what Linux distro you’ve picked, it might have Apache already installed. We don’t want that so uninstall it with:

sudo apt-get remove apache2

3. Install NGINX:

 sudo apt-get install nginx

4. NGINX should start automatically, but just in case, you can start it manually with:

sudo systemctl start nginx

5. Confirm that NGINX is up and running by opening your browser and visiting your IP address. You should be able to get the default page to display by visiting your loopback IP address (127.0.0.1) or the actual IP assigned to your device (available by running the command ‘hostname -I’ in the terminal):

6. Good! Now we have NGINX up and running! If you don’t have a backend web server running yet, then we’re done since you don’t have anywhere for us to send traffic. Come back to this point when you do. But if you do have a web server for us to proxy traffic to/from, continue on!

Configuring the Reverse Proxy:

So you’ve made it this far and you now have an NGINX server running. Let’s set up the reverse proxy part to make this an NGINX reverse proxy and not just a simple NGINX web server:

1. Go to our NGINX sites-available directory:

cd /etc/nginx/sites-available/

2. Create the configuration file. You’ll eventually accumulate a lot of these, so I recommend naming it based on the site that you’re reverse proxying so you can easily find it again:

sudo nano example.com.conf

3. In nano, add the following:

server {
	listen 80;
	server_name: example.com
	location / {
	proxy_pass http://192.x.x.2;
	}
}

server_name is going to contain the domain name of the website clients are going to be requesting. proxy_pass is going to be the local (internal) IP address of the web server that you’re forwarding traffic to. You can also specify a particular port if your web server is running on a non-standard port (example: proxy_pass http://192.x.x.2:82300).

4. For NGINX to actually serve your site with your new configuration, you need to link it to /sites-enabled/ with:

ln -s /etc/nginx/sites-available/example.com.conf /etc/nginx/sites-enabled/example.com.conf

5. Test your configuration to make sure you aren’t getting any errors:

sudo nginx -t

6. Reload NGINX to tell it that the configuration has been updated:

sudo systemctl reload nginx

That’s all there is to it! In future posts, I’ll cover how to setup that upstream web server, how to configure the DNS for your domain, and port forwarding so that you can access your sites on the internet.

As always, feel free to comment if you run into any problems or need help!

Category: Servers | LEAVE A COMMENT