Connecting an Arduino to the Internet

Building Simple Network Devices

At a very basic level, the Internet of Things are simply sensors, actuators and computers connected through a network. In this eco-system, computers either gather real-word data through sensors, and/or alter the real-world by sending instructions to actuators. Regardless of complexity, IoT systems can often be reduced to these three common denominators.

Sensors and actuators are essentially network embedded devices--single-purpose computing systems connected to a network. These devices have hardware and operating systems optimized to run a single application. In this post we will look into the basic elements of a network embedded device by building a simple sensor and actuator device using an Arduino Uno and Ethernet Shield. Although things are becoming predominantly wireless, we are using the Ethernet to illustrate the simple basic building blocks. Well actually, I just found an Ethernet Shield that I can't recall ordering and decided to just use it for a blog post.

Connecting the Dots

A network embedded device can be broken down into the following parts:

• Microcontroller: a small computer with a single integrated circuit that runs the device application software.
• Network Interface: Connects the device to the network so it can communicate with computers or other embedded devices
• Application Component: In an IoT context, this is either a sensor that provides real-word data or an actuator that executes real-world action.

In some cases, both the the microntroller and the network interface are integrated on a single board. In this post, we will use an Arduino as the microcontroller and the Ethernet shield as the network interface. We will go through two examples with different application components: a DTH11 Humidity and Temperature Sensor for a sensor server device and standard 16X2 LCD screen for the client actuator device.

Arduino Ethernet Shield

The Ethernet shield  connects to the Arduino Uno using the Serial Peripheral Interface (SPI), a synchronous serial communication protocol commonly used to connect microcontrollers with other peripheral components. It follows a Slave/Master model as shown below:

The Master component  (i.e. the microcontroller) sends a timing signal through a Clock (SCK) line to synchronize communication between two separate data lines. Depending on the clock cycle,  Slave components either receive data from the Master through a Master Out Slave In (MOSI) line, or send out data to the Master through a Master In Slave Out (MISO) line. The master can control several slave modules through multiple Slave Select (SSL) lines.  To activate a Slave component the SSL line connected to that device is set to high, and deactivated by setting it to low. Only one slave device is activated at any given point.

On the Arduino Uno, the SPI lines are as follows:

- SCK - Pin 13

- MISO - Pin 12

- MOSI - Pin 11

We will use the Arduino as a Master component and connect the Ethernet shield as a Slave. If set as the Master component, any digital pin set to output on the Arduino can be used as an SS line. However, Slave devices have dedicated SS lines. The Ethernet shield also uses  Pins 13, 12 and 11 as the SCK, MISO and MOSI lines respectively. It uses Pin 10 as the SS line to the Ethernet interface, and Pin 4 as the SS line to the SD card interface (which we won't be using in our examples.) The illustration below shows how the Arduino and the Ethernet shield are connected:

Check the, the Arduino website for information on the Ethernet. 

We will use this shield to build simple network devices that communicates via HTTP, the ubiquitous protocol that connects the World Wide Web. Attach the shield to the Arduino and connect it to a hub or router with an active connection to the Internet.

Network Sensor Device

Our first device is a web server that sends the temperature and humidity reading from a DTH11 sensor to web page. Data from the DHT11 is collected by the Arduino Uno that connects to the Internet via the Ethernet shield.

DTH11 Sensor Pins

The DHT11 sensor has 4 pins, the first is the power supply (VCC) pin that works between 3.5 to 5.5 volts, the second is the data pin, and the fourth and last is the Ground pin. The third pin is an empty pin and I don't really know why it's even there. If you are using a DHT11 on a breakout board it will only have three pins so you don't need to bother with the third pin.

Assuming the the Ethernet shield is attached to the Arduino, connect the DHT11 as follows:

• 5v power pin on the Ethernet shield to the first pin on the DHT11.
• Any Ground pin on the Ethernet shield to the last pin on the DHT11.
• Pin 9 on the Ethernet shield to the second pin of the DHT11.

The data pin communicates using single-wire bi-directional serial protocol. Instead of a separate clock line, it uses one line for the synchronization signal and the data transmission as follows:

• The Arduino sends a signal to the data pin and sets the sensor into a high-power state.
• The DHT11 waits for the signal to end, and sends the humidity and temperature data.
• It returns to a low-power state and waits for the next signal from the Arduino.

Upload the following code to the Arduino:

temp_humidity_sensor.ino

 [code]  
 /*  
 Basic network embedded device webserver that sends the output of a  
 DHT11 Temperature and Humidity Sensor on HTTP request.   
 */  
   
 #include <SimpleDHT.h>  
 #include <SPI.h>  
 #include <Ethernet.h>  
   
 // Enter a MAC address for your controller below.  
 // Newer Ethernet shields have a MAC address printed on a sticker on the shield  
 byte mac[] = {  
  0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x02  
 };  
   
 //DHT11 Input Pin  
 int pinDHT11 = 9;  
   
 //Initialize DHT11 Sensor  
 SimpleDHT11 dht11;  
   
 //initialize variable for DHT11 sensor  
 byte temperature = 0;  
 byte humidity = 0;  
 byte data[40] = {0};  
   
   
 // Initialize the a server that listens for a request  
 // on port 80  
 EthernetServer server(80);  
   
 void setup() {  
    
  // Open serial connection  
  Serial.begin(9600);  
     
   // Establish Ethernet connection:  
  if (Ethernet.begin(mac) == 0) {  
   // if can't connect print error msg and loop foever:  
   Serial.println("Failed to configure Ethernet using DHCP");  
   for (;;)  
   ;  
  }  
  // print your local IP address:  
  printIPAddress();  
   
  //start server  
  server.begin();  
      
 }  
   
 void loop() {  
  //reconnect when network is disconnected  
  networkReconnect();  
   
  //read DHT11 Data  
  if (dht11.read(pinDHT11, &temperature, &humidity, data)) {  
   Serial.println("Read DHT11 failed");  
   return;  
   }  
  else {  
   
    Serial.println("");  
    Serial.print("Sample OK: ");  
    Serial.print((int)temperature);   
    Serial.print(" *C, ");  
    Serial.print((int)humidity);   
    Serial.println(" %");    
   }  
   
  // listen for incoming clients  
  EthernetClient client = server.available();   
  if (client) {  
   
    // notify serial each server request  
    Serial.println("client connection");  
   
    // length of the incoming text line  
    int lineLength = 0;   
   
    //run while client is connected  
    while (client.connected()) {  
   
     //check if a request has been made  
     if (client.available()) {  
   
      // read one character from the request data stream    
      char thisChar = client.read();  
   
        
      // Check for end of header (blank line)  
      if (thisChar == '\n' && lineLength < 1) {  
        // send a standard http response header  
        makeResponse(client);  
        break;  
        }  
       
      // check for end of the line and reset linelength 0 ,  
      if (thisChar == '\n' || thisChar == '\r') {  
       lineLength = 0;  
      }  
        
      else {  
       // for any other character, increment the line length:  
       lineLength++;  
       }  
       Serial.write(thisChar);  
      }  
     }  
     
   // close the connection:  
   client.stop();  
   
  }  
   
  // DHT11 sampling rate is 1HZ.  
  delay(1000);  
 }  
   
 void networkReconnect() {  
   
   //reconnect if ethernet is disconnected  
  switch (Ethernet.maintain())  
  {  
   case 1:  
    //renewed fail  
    Serial.println("Error: renewed fail");  
    break;  
   
   case 2:  
    //renewed success  
    Serial.println("Renewed success");  
   
    //print your local IP address:  
    printIPAddress();  
    break;  
   
   case 3:  
    //rebind fail  
    Serial.println("Error: rebind fail");  
    break;  
   
   case 4:  
    //rebind success  
    Serial.println("Rebind success");  
   
    //print your local IP address:  
    printIPAddress();  
    break;  
   
   default:  
    //nothing happened  
    break;  
   
  }  
 }  
   
 // function for printing local IP address to serial  
 void printIPAddress()  
 {  
  Serial.print("My IP address: ");  
  for (byte thisByte = 0; thisByte < 4; thisByte++) {  
   // print the value of each byte of the IP address:  
   Serial.print(Ethernet.localIP()[thisByte], DEC);  
   Serial.print(".");  
  }  
   
  Serial.println();  
 }  
   
 // function for server response  
 void makeResponse(EthernetClient thisClient) {  
    
  thisClient.print("HTTP/1.1 200 OK\n");  
  thisClient.print("Content-Type: text/html\n\n");  
  thisClient.println("<html><head></head><body>\n");  
  thisClient.print("<h1>Hello! I'm an Arduino Web Server!</h1><br>");  
  thisClient.print("<b>Current Room Temperature:</b> ");  
  thisClient.print((int)temperature);  
  thisClient.print(" *C<br>");  
  thisClient.print("<b>Current Room Humidity:</b> ");  
  thisClient.print((int)humidity);  
  thisClient.print(" %<br>");   
  thisClient.println("</body></html>\n");  
  }  
   
 [/code]  

There are several Arduino libraries for the DHT11 and simpleDHT is used for this example. Follow this link for installation instructions.

We include the SPI and Ethernet libraries to access the networking functions of the Ethernet shield, and simpleDHT libary to access the sensor. We declare global variables starting with the MAC address of the shield. This address is typically printed on the shield, but an arbitrary address can also be used as shown in the code. We assign pin 9 as our data pin, and create additional variables for the humidity and sensor readings, as well as the raw 40-bit data sent by the sensor.

Managing network connection and client requests

• We instantiate an EthernetServer object to handle client requests, setting the port number parameter to 80 (The port number actually defaults to 80 unless it's explicitly set).  We start the program by opening a serial connection for sending debug information to the Serial Monitor.
• An Ethernet connection is initiated by passing the MAC address to the Ethernet.begin() method. It will establish a connection through the hub or router and request for an I.P address via DHCP.  It returns 0 if the if it cannot establish a connection. This is handled by sending an error message to the the Serial Monitor and suspending the program through an endless loop.  Otherwise, if a network connection is established it calls the printIPAddress() function and sends the device I.P. address to the Serial Monitor. This function obtains the local I.P address via Ethernet.localIP() method,
• The assigned I.P address is leased to the device for a specified amount of time. It's necessary to consistently check, and renew, the address if it has expired. This is done at the start of the program loop by calling the networkReconnect() function. It uses Ethernet.maintain() method to renew the I.P address.  The method returns the following byte that the function handles accordingly with a switch statement:

• 0: nothing happened
• 1: renew failed
• 2: renew success
• 3: rebind fail
• 4: rebind success

• We start listening for client requests by calling server.begin() and checking for a client connection with server.available(), It returns an object representing the client that is connected to the server.  The client.connected() method checks if the client connection is still active. While the client is connected, we check for incoming data from the client, which can then be read per byte with the client.read() method.
• Each time a web browser request is made, it sends an HTTP header that contains the request parameters and and ends with an empty line. The server response is based on the values of the request parameters. In this simple example, we just send the server response after the end of the header (blank line) has been read.
• The makeResponse() function takes a Client object and sends an HTTP response using the print() and println(). It starts the response with a header that ends with a blank line, followed by the HTML content of the response, which contains the values read from the DTH11 sensor, which we will discuss next.

Reading the Sensor

The DHT11 library has a read() method that takes three parameters: the pin where the data line is connected, a variable for the temperature reading, a variable for the humidity reading, and a last variable for the 40-bit raw data. We won't be using the raw data but we still assign a variable just to be consistent when passing parameters to the read() method. The temperature and humidity values are stored in assigned variables with the same names, and are are sent with of the server response in the makeResponse() function.

Plug the Ethernet shield to a a router or hub once the the circuit has been assembled and the code uploaded to the Arduino. Open the Serial Monitor to check the debug messages.  If all went well, the Serial Monitor will display the local I.P. address of your device. Go to the address using a web browser and the current temperature and humidity will be displayed.  The code can be modified to respond with JSON or other data format instead of HTML.

Click to Zoom

Network Actuator Device

The next example is a client device that retrieves the current local time from an API provided by timzonedb.com, and displays it on 16x2 LCD screen.  You need to register an account in timezonedb.com to access the API. Take note of the API key of your registered account. 

We will use a simple monochrome alphanumeric display that runs on a Hitachi HD44780 controller. It has parallel data and control interface as follows:

• 8 Data pins (D0 -D7)
• A Register Select (RS) pin to switch between sending either data for display or instruction sets.
• A Read/Write (R/W) pin to switch to either reading mode or writing mode
• An Enable pin to enable/disable writing to the either the display or instruction registers
• There is also +5V power supply (VCC) pin with a respective Ground pin, another pin to control the display contrast (VDD), and two pins to the turn on/off the LCD backlight.

Again, assuming that the Ethernet Shield is attached to the Arduino, connect the LCD display to the shield as follows:

• LCD RS pin digital to Arduino pin 12
• LCD Enable pin to Arduino digital pin 11
• LCD 4 pin to Arduino digital pin 5
• LCD 5 pin to Arduino digital pin 4
• LCD 6 pin to Arduino digital pin 3
• LCD 7 pin to Arduino digital pin 2
• Connect the potentiometer to the breadboard power and ground rail and the middle wiper pin to the LCD VDD pin to control the contrast.
• Turn on the backlight by connecting the LCD positive LED pin to the power rail with a 220 ohm pull-up resistor and the LCD negative LED pin to the ground rail.

Upload the following Sketches code on to Arduino:

displayLocalTime.ino

 // Simple Network Embedded Device  
 // Extracts date and time from timezonedb.com  
 // and displays on LCD screen  
   
 #include <ArduinoJson.h> //JSON Library  
 #include <Ethernet.h> //Ethernet Library  
 #include <SPI.h> //SPI Library  
 #include <LiquidCrystal.h> // Liquid Crystal Library  
   
 // initialize the LCD library with RS pin on 8, E pin on 9, and pin 5,4,3,2 as output  
 LiquidCrystal lcd(9, 8, 5, 4, 3, 2);  
   
 // initialize http client  
 EthernetClient client;  
   
 const char* server = "api.timezonedb.com"; // API server address  
 // API endpoint  
 const char* resource = "/v2/get-time-zone?key=G09GM1DXFAH7&format=json&by=zone&zone=Asia/Manila";   
 const unsigned long BAUD_RATE = 9600;   // serial connection speed  
 const unsigned long HTTP_TIMEOUT = 10000; // max respone time from server  
 const size_t MAX_CONTENT_SIZE = 512;    // max size of the HTTP response  
   
 // data type to extract from endpoint  
 struct LocalTime {  
  char country[32];  
  char time[32];  
 };  
   
 //Initialize program  
 void setup() {  
    
  //initialize serial library  
  initSerial();  
   
  //initialize Ethernet library  
  initEthernet();  
     
  // set up the LCD's number of columns and rows:  
  lcd.begin(16, 2);  
    
 }  
   
 void loop() {  
  //reconnect if Ethernet is disconnected  
  networkReconnect();  
     
  if (connect(server)) {  
   if (sendRequest(server,resource) && skipResponseHeaders()) {  
    LocalTime localTime;  
    if (readReponseContent(&localTime)) {  
     printUserData(&localTime);  
     printLCD(&localTime);  
    }  
   }  
  }  
   
  //disconnect from server  
  disconnect();  
   
  //wait before next reconnection  
  wait();  
 }  
   
 // Initialize Serial port  
 void initSerial() {  
  Serial.begin(BAUD_RATE);  
  while (!Serial) {  
   ; // wait for serial port to initialize  
  }  
  Serial.println("Serial ready");  
 }  
   
 // Initialize Ethernet library  
 void initEthernet() {  
  byte mac[] = {0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED};  
  if (!Ethernet.begin(mac)) {  
   Serial.println("Failed to configure Ethernet");  
   return;  
  }  
  Serial.println("Ethernet ready");  
  printIPAddress();  
    
  delay(1000);  
 }  
   
 // Print local IP address to serial  
 void printIPAddress()  
 {  
  Serial.print("IP address: ");  
  for (byte thisByte = 0; thisByte < 4; thisByte++) {  
   // print the value of each byte of the IP address:  
   Serial.print(Ethernet.localIP()[thisByte], DEC);  
   Serial.print(".");  
  }  
   
  Serial.println();  
 }  
   
   
 //reconnect if ethernet is disconnected  
 void networkReconnect() {  
   
  switch (Ethernet.maintain())  
  {  
   case 1:  
    //renewed fail  
    Serial.println("Error: renewed fail");  
    break;  
   
   case 2:  
    //renewed success  
    Serial.println("Renewed success");  
   
    //print your local IP address:  
    printIPAddress();  
    break;  
   
   case 3:  
    //rebind fail  
    Serial.println("Error: rebind fail");  
    break;  
   
   case 4:  
    //rebind success  
    Serial.println("Rebind success");  
   
    //print your local IP address:  
    printIPAddress();  
    break;  
   
   default:  
    //nothing happened  
    break;  
   
  }  
 }  
   
 // Open connection to the HTTP server  
 bool connect(const char* hostName) {  
  Serial.print("Connect to ");  
  Serial.println(hostName);  
   
  bool ok = client.connect(hostName, 80);  
   
  Serial.println(ok ? "Connected" : "Connection Failed!");  
  return ok;  
 }  
   
 // Send the HTTP GET request to the server  
 bool sendRequest(const char* host, const char* resource) {  
  Serial.print("GET ");  
  Serial.println(resource);  
   
  client.print("GET ");  
  client.print(resource);  
  client.println(" HTTP/1.0");  
  client.print("Host: ");  
  client.println(host);  
  client.println("Connection: close");  
  client.println();  
   
  return true;  
 }  
   
 // Strip HTTP headers  
 bool skipResponseHeaders() {  
  // HTTP headers end with an empty line  
  char endOfHeaders[] = "\r\n\r\n";  
   
  client.setTimeout(HTTP_TIMEOUT);  
  bool ok = client.find(endOfHeaders);  
   
  if (!ok) {  
   Serial.println("No response or invalid response!");  
  }  
   
  return ok;  
 }  
   
 // Parse the JSON from the input string and extract the relevant values  
 // Here is the JSON we need to parse  
 bool readReponseContent(struct LocalTime* localTime) {  
  // Compute optimal size of the JSON buffer according to what we need to parse.  
  // See https://bblanchon.github.io/ArduinoJson/assistant/  
  const size_t BUFFER_SIZE = JSON_OBJECT_SIZE(13) + 215;  
   
  // Allocate a temporary memory pool  
  DynamicJsonBuffer jsonBuffer(BUFFER_SIZE);  
   
  JsonObject& root = jsonBuffer.parseObject(client);  
   
  if (!root.success()) {  
   Serial.println("JSON parsing failed!");  
   return false;  
  }  
   
  // Here were copy the strings we're interested in  
  strcpy(localTime->country, root["countryName"]);  
  strcpy(localTime->time, root["formatted"]);  
    
  return true;  
 }  
   
 // Print the data extracted from the JSON  
 void printUserData(const struct LocalTime* localTime) {  
  Serial.print("Country = ");  
  Serial.println(localTime->country);  
  Serial.print("Time = ");  
  Serial.println(localTime->time);  
 }  
   
 // Output to LCD  
 void printLCD(const struct LocalTime* localTime) {  
  lcd.print(localTime->country);  
  lcd.setCursor(0, 1);  
  lcd.print(localTime->time);  
 }  
   
   
 // Close the connection with the HTTP server  
 void disconnect() {  
  Serial.println("Disconnect");  
  client.stop();  
 }  
   
 // Pause   
 void wait() {  
  Serial.println("Wait 30 seconds");  
  delay(30000);  
 }  

Most of the code for managing the Ethernet and Serial interface is reused from the previous example. We also include the ArduinoJSON library by Bennot Blanchon for parsing the JSON response from API and the LiquidCrystal library which manages the RS, Enable and data pins of the the LCD display. The VDD and back light pins on the LCD pins are set via analog inputs directly from the breadboard.

We declare constants  for the API server address, the API endpoint (use your API key as value for 'key'), baud rate for the serial connection. 

Making an HTTP Request

• We instantiate an EthernetClient object to handle HTTP requests.
• We initiate Serial and Ethernet connections, using initSerial() and initEthernet() functions respectively. Both functions reused the code from the previous example. 
• The main loop starts by by calling networkReconnect() to check the lease status of the DHCP assigned I.P. address. Then we attempt to establish a client connection by passing the server address to  the connect() function, which accepts a valid host name. It uses the connect() method of the EthernetClient class to make a connection. The method returns True if the connection was successfully established.
• If a connection has been established, an API request is sent by passing the server and endpoint string to sendRequest(), and validating it with skipHeaderResponse(). sendRequest() sends a GET request using the print and println methods of the Ethernet Client class. skipHeaderResponse() validates the response by finding the two blank lines that indicates the end of a HTTP header.
• If the server returns a valid HTTP response we parse the JSON content by calling readResponseContent() and passing it a pointer to LocalTime.

Parsing JSON

The ArduinoJSON library preallocates memory for the JSON object. It supports allocation of a fixed memory pool using the StaticJsonBuffer class, or dynamic memory allocation via the DynamicJsonBuffer class. ArduinoJSON Assistant is a browser-based utility for determining the amount of memory needed for a given JSON object tree.  It takes a JSON  input and returns the expression for computing the buffer size. It also gives a code snippet for a parsing program. Refer to the documentation page for a more details on the memory model.

Here's the JSON response from the API end-point: 

http://api.timezonedb.com/v2/get-time-zone?key=G09GM1DXFAH7&format=json&by=zone&zone=Asia/Manila

{

"status":"OK",

"message":"",

"countryCode":"PH",

"countryName":"Philippines",

"zoneName":"Asia\/Manila",

"abbreviation":"+08",

"gmtOffset":28800,

"dst":"0",

"dstStart":275151600,

"dstEnd":0,

"nextAbbreviation":"",

"timestamp":1510066974,

"formatted":"2017-11-07 15:02:54"

}

Which gives us the following expression in ArduinoJSON Assistant:

Parsing of the JSON content from the API response is handled by the readResponseContent() function:

• The expression returned by ArduinoJSON Assistant is declared as the value of the BUFFER_SIZE constant,  A temporary memory is allocated by passing this value to an instance of DynamicJsonBuffer.
• Once memory has been allocated, we pass the client object to the parseJsonObject() method of DynamicJsonBuffer to parse the content of the server response which returns an instance of the JsonObject class that represents the object tree.
• The success() method of of the JsonObject class returns true if it contains a valid JSON object tree.
• If a valid JSON object was returned, we copy the values of 'countryName' and 'formatted' from the JSON tree to  localTime->country' and localTime->time respectively.

Displaying Data on the LCD

• The LCD is initialized by passing the RS, Enable and output pins to an instance of the LiquidCrystal class.
• The number of rows and columns is set with begin() method which is called at setup().
• In the main program loop, we display the relevant data to the LCD by passing a pointer to LocalTime  to the displayLCD() function, if readResponseContent() returns true.
• displayLCD () displays the values using print() method. The cursor() method is used to move to the position where the line should start.

The if we connect the Ethernet Shield to a hub or router theLCD should display the local time from the API endpoint. The contrast is adjusted by using the potentiometer.

Microcontrollers have relatively small memory size and can't handle large or complex JSON objects. I originally intended to display a 'chance of rain' forecast from Weather Underdground, but the API returns a large set of forecast data as a multi-dimensional JSON tree which crashed the Arduino Uno. In cases like this, you need to build another web application hosted on an external cloud server to parse and extract a small sub-set of data from a large JSON object. This data subset is then exposed through a REST API as a simple JSON tree that can be handled by microcontrollers.

These examples are proof of concepts meant to illustrate the basic building blocks of network sensor and actuator devices. For actual projects, it would be more practical to use a microcontoller with an integrated esp8266 wifi interface such as the Adafruit Huzzah. If an Ethernet is required for performance or security, an ENC28J60 module would also be a good option.

Once again, if you have any comments, spot any errors, say hello, or simply rant, you are welcome to do that at the comments section below.