In this chapter we will introduce and examine the use of Ethernet networking with Arduino over local networks and the greater Internet.

It will be assumed that you have a basic understanding of computer networking, such as the knowledge of how to connect computers to a hub/router with RJ45 cables, what an IP and MAC address is, and so on. Furthermore, here is a good quick rundown about Ethernet.

Getting Started

You will need an Arduino Uno or compatible board with an Ethernet shield that uses the W5100 Ethernet controller IC (pretty much all of them):

Furthermore you will need to power the board via the external DC socket – the W5100 IC uses more current than the USB power can supply. A 9V 1.5A plug pack/wall wart will suffice.

Finally it does get hot – so be careful not to touch the W5100 after extended use. In case you’re not sure – this is the W5100 IC:

Once you have your Ethernet-enabled Arduino, and have the external power connected – it’s a good idea to check it all works. Open the Arduino IDE and select File > Examples > Ethernet > Webserver. This loads a simple sketch which will display data gathered from the analogue inputs on a web browser. However don’t upload it yet, it needs a slight modification.

You need to specify the IP address of the Ethernet shield – which is done inside the sketch. This is simple, go to the line:

IPAddress ip(192,168,1, 177);

And alter it to match your own setup. For example, in my home the router’s IP address is 10.1.1.1, the printer is 10.1.1.50 and all PCs are below …50. So I will set my shield IP to 10.1.1.77 by altering the line to:

IPAddress ip(10,1,1,77);

byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

However if you only have one shield just leave it be. There may be the very, very, statistically rare chance of having a MAC address the same as your existing hardware, so that would be another time to change it.

Once you have made your alterations, save and upload the sketch. Now open a web browser and navigate to the IP address you entered in the sketch, and you should be presented with something similar to the following:

What’s happening? The Arduino has been programmed to offer a simple web page with the values measured by the analogue inputs. You can refresh the browser to get updated values.

At this point – please note that the Ethernet shields use digital pins 10~13, so you can’t use those for anything else. Some Arduino Ethernet shields may also have a microSD card socket, which also uses another digital pin – so check with the documentation to find out which one.

Nevertheless, now that we can see the Ethernet shield is working we can move on to something more useful. Let’s dissect the previous example in a simple way, and see how we can distribute and display more interesting data over the network. For reference, all of the Ethernet-related functions are handled by the Ethernet Arduino library. If you examine the previous sketch we just used, the section that will be of interest is:

 for (int analogChannel = 0; analogChannel < 6; analogChannel++) 
          {
            int sensorReading = analogRead(analogChannel);
            client.print("analog input ");
            client.print(analogChannel);
            client.print(" is ");
            client.print(sensorReading);
            client.println("<br />");       
          }
          client.println("</html>");

Hopefully this section of the sketch should be familiar – remember how we have used serial.print(); in the past when sending data to the serial monitor box? Well now we can do the same thing, but sending data from our Ethernet shield back to a web browser – on other words, a very basic type of web page.

However there is something you may or may not want to  learn in order to format the output in a readable format – HTML code. I am not a website developer (!) so will not delve into HTML too much.

However if you wish to serve up nicely formatted web pages with your Arduino and so on, here would be a good start. In the interests of simplicity, the following two functions will be the most useful:

client.print(" is ");

Client.print (); allows us to send text or data back to the web page. It works in the same way as serial.print(), so nothing new there. You can also specify the data type in the same way as with serial.print(). Naturally you can also use it to send data back as well. The other useful line is:

client.println("<br />");

which sends the HTML code back to the web browser telling it to start a new line. The part that actually causes the carriage return/new line is the <br /> which is an HTML code (or “tag”) for a new line. So if you are creating more elaborate web page displays, you can just insert other HTML tags in the client.print(); statement.

If you want to learn more about HTML commands, here’s a good tutorial site. Finally – note that the sketch will only send the data when it has been requested, that is when it has received a request from the web browser.

Accessing your Arduino over the Internet

So far – so good. But what if you want to access your Arduino from outside the local network?

You will need a static IP address – that is, the IP address your internet service provider assigns to your connection needs to stay the same. If you don’t have a static IP, as long as you leave your modem/router permanently swiched on your IP shouldn’t change. However that isn’t an optimal solution.

If your ISP cannot offer you a static IP at all, you can still move forward with the project by using an organisation that offers a Dynamic DNS. These organisations offer you your own static IP host name (e.g. mojo.monkeynuts.com) instead of a number, keep track of your changing IP address and linking it to the new host name. From what I can gather, your modem needs to support (have an in-built client for…) these DDNS services. As an example, two companies are No-IP andDynDNS.com. Please note that I haven’t used those two, they are just offered as examples.

Now, to find your IP address… usually this can be found by logging into your router’s administration page – it is usually 192.168.0.1 but could be different. Check with your supplier or ISP if they supplied the hardware. For this example, if I enter 10.1.1.1 in a web browser, and after entering my modem administration password, the following screen is presented:

What you are looking for is your WAN IP address, as you can see in the image above. To keep the pranksters away, I have blacked out some of my address.

The next thing to do is turn on port-forwarding. This tells the router where to redirect incoming requests from the outside world. When the modem receives such a request, we want to send that request to the port number of our Ethernet shield. Using the:

EthernetServer server(125);

function in our sketch has set the port number to 125. Each modem’s configuration screen will look different, but as an example here is one:

So you can see from the line number one in the image above, the inbound port numbers have been set to 125, and the IP address of the Ethernet shield has been set to 10.1.1.77 – the same as in the sketch.

After saving the settings, we’re all set. The external address of my Ethernet shield will be the WAN:125, so to access the Arduino I will type my WAN address with :125 at the end into the browser of the remote web device, which will contact the lonely Ethernet hardware back home.

Furthermore, you may need to alter your modem’s firewall settings, to allow the port 125 to be “open” to incoming requests. Please check your modem documentation for more information on how to do this.

Now from basically any Internet connected device in the free world, I can enter my WAN and port number into the URL field and receive the results. For example, from a phone when it is connected to the Internet via LTE mobile data:

So at this stage you can now display data on a simple web page created by your Arduino and access it from anywhere with unrestricted Internet access. With your previous Arduino knowledge you can now use data from sensors or other parts of a sketch and display it for retrieval.

Displaying sensor data on a web page

As an example of displaying sensor data on a web page, let’s use an inexpensive and popular temperature and humidity sensor – the DHT22. You will need to install the DHT22 Arduino library which can be found on this page. If this is your first time with the DHT22, experiment with the example sketch that’s included with the library so you understand how it works.

Connect the DHT22 with the data pin to Arduino D2, Vin to the 5V pin and GND to … GND.

Now for our sketch – to display the temperature and humidity on a web page. If you’re not up on HTML you can use online services such as this to generate the code, which you can then modify to use in the sketch.

In the example below, the temperature and humidity data from the DHT22 is served in a simple web page:

#include <SPI.h>
#include <Ethernet.h>

// for DHT22 sensor
#include "DHT.h"
#define DHTPIN 2
#define DHTTYPE DHT22

// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = {   0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
IPAddress ip(10,1,1,77);

// Initialize the Ethernet server library
// with the IP address and port you want to use 
// (port 80 is default for HTTP):
EthernetServer server(125);
DHT dht(DHTPIN, DHTTYPE);

void setup() 
{
  dht.begin();
 // Open serial communications and wait for port to open:
  Serial.begin(9600);
   while (!Serial) {
    ; // wait for serial port to connect. Needed for Leonardo only
  }
  // start the Ethernet connection and the server:
  Ethernet.begin(mac, ip);
  server.begin();
  Serial.print("server is at ");
  Serial.println(Ethernet.localIP());
}

void loop() 
{
  // listen for incoming clients
  EthernetClient client = server.available();
  if (client) {
    Serial.println("new client");
    // an http request ends with a blank line
    boolean currentLineIsBlank = true;
    while (client.connected()) {
      if (client.available()) {
        char c = client.read();
        Serial.write(c);
        // if you've gotten to the end of the line (received a newline
        // character) and the line is blank, the http request has ended,
        // so you can send a reply
        if (c == 'n' && currentLineIsBlank) 
        {
          // send a standard http response header
          client.println("HTTP/1.1 200 OK");
          client.println("Content-Type: text/html");
          client.println("Connection: close");  // the connection will be closed after completion of the response
	  client.println("Refresh: 30");  // refresh the page automatically every 30 sec
          client.println();
          client.println("<!DOCTYPE HTML>");
          client.println("<html>");

          // get data from DHT22 sensor
          float h = dht.readHumidity();
          float t = dht.readTemperature();
          Serial.println(t);
          Serial.println(h);

          // from here we can enter our own HTML code to create the web page
          client.print("<head><title>Office Weather</title></head><body><h1>Office Temperature</h1><p>Temperature - ");
          client.print(t);
          client.print(" degrees Celsius</p>");
          client.print("<p>Humidity - ");
          client.print(h);
          client.print(" percent</p>");
          client.print("<p><em>Page refreshes every 30 seconds.</em></p></body></html>");
          break;
        }
        if (c == 'n') {
          // you're starting a new line
          currentLineIsBlank = true;
        } 
        else if (c != 'r') {
          // you've gotten a character on the current line
          currentLineIsBlank = false;
        }
      }
    }
    // give the web browser time to receive the data
    delay(1);
    // close the connection:
    client.stop();
    Serial.println("client disonnected");
  }
}

It is a modification of the IDE’s webserver example sketch that we used previously – with a few modifications. First, the webpage will automatically refresh every 30 seconds – this parameter is set in the line:

client.println("Refresh: 30");  // refresh the page automatically every 30 sec

… and the custom HTML for our web page starts below the line:

// from here we can enter our own HTML code to create the web page

You can then simply insert the required HTML inside client.print() functions to create the layout you need.

Finally – here’s an example screen shot of the example sketch at work:

You now have the framework to create your own web pages that can display various data processed with your Arduino.

Conclusion

I hope you enjoyed making this or at least reading about it. If you find this sort of thing interesting, please consider ordering one or more of my books from amazon.

And as always, have fun and make something.

To keep up to date with new posts at tronixstuff.com, please subscribe to the mailing list in the box on the right, or follow us on twitter @tronixstuff.

When first learning about and building electronic circuits, the first things all of us come across are passive components such as resistors, capacitors, and inductors. These have easily-understandable properties and are used in nearly all circuits in some way or another. Eventually we’ll move on to learning about active components like transistors, but there’s a fourth passive circuit component that’s almost never encountered. Known as the memristor, this mysterious device is not quite as intuitive as the other three, so [Andrew] created an Arduino shield to investigate their properties.

Memristors relate electric charge and magnetic flux linkage, which means that their resistance changes based on the current that passes through them. As their name implies, this means they have memory, and retain their properties even after power is removed. [Andrew] is testing three different memristors, composed of tungsten, carbon, and chromium, using this specialized test set. The rig is based on an Arduino Uno and has a few circuit components that can be used as references and generates data on the behavior of the memristors under various situations.

The memristors used here do exhibit expected behavior when driven with positive voltage signals, but did exhibit a large amount of variability when voltage was applied in a negative direction. [Andrew] speculates that using these devices for storage would be difficult and would likely require fairly bespoke applications for each type. But as the applications for these seemingly bizarre circuit components increase, we expect them to improve much like any other passive component.

While it can be straightforward to distill water to high purity, this is rarely the best method for producing water for useful purposes. Even drinking water typically needs certain minerals in it, plants may need a certain pH, and wastewater systems have a whole host of other qualities that need to be measured. Measuring water quality is a surprisingly complex endeavor as a result and often involves a wide array of sensors, much like this water quality meter from [RowlesGroupResearch].

The water quality meters that they are putting to use are typically set up in remote locations, without power, and are targeting natural bodies of water and also wastewater treatment plants. Temperature and pH are simple enough to measure and grasp, but this device also includes sensors for total dissolved solids (TDS) and turbidity which are both methods for measuring various amounts and types of particles suspended in the water. The build is based around an Arduino so that it is easy for others to replicate, and is housed in a waterproof box with a large battery, and includes data logging to an SD card in order to make it easy to deploy in remote, outdoor settings and to gather the data at a later time.

The build log for this device also goes into detail about all of the steps needed to set this up from scratch, as well as a comprehensive bill of materials. This could be useful in plenty of professional settings such as community wastewater treatment facilities but also in situations where it’s believed that industrial activity may be impacting a natural body of water. For a water quality meter more focused on drinking water, though, we’d recommend this build that is trained on its own neural network.

There are plenty of problems that are easy for humans to solve, but are almost impossibly difficult for computers. Even though it seems that with modern computing power being what it is we should be able to solve a lot of these problems, things like identifying objects in images remains fairly difficult. Similarly, identifying specific sounds within audio samples remains problematic, and as [Eivind] found, is holding up a lot of medical research to boot. To solve one specific problem he created a system for counting coughs of medical patients.

This was built with the idea of helping people with chronic obstructive pulmonary disease (COPD). Most of the existing methods for studying the disease and treating patients with it involves manually counting the number of coughs on an audio recording. While there are some software solutions to this problem to save some time, this device seeks to identify coughs in real time as they happen. It does this by training a model using tinyML to identify coughs and reject cough-like sounds. Everything runs on an Arduino Nano with BLE for communication.

While the only data the model has been trained on are sounds from [Eivind], the existing prototypes do seem to show promise. With more sound data this could be a powerful tool for patients with this disease. And, even though this uses machine learning on a small platform, we have seen before that Arudinos are plenty capable of being effective machine learning solutions with the right tools on board.

We’re suckers for alternative data transmission methods, and [Georgi Gerganov]’s ggwave made us smile. At its core, it’s doing what the phone modems of old used to do – sending data encoded as different audio tones. But GGwave does this with sophistication!

It splits the data into four-bit chunks, and uses 16 different frequency offsets to represent each possible value. But for each chunk, these offsets are added to one of six different base frequencies, which allows the receiving computer to tell which chunk it’s in. It’s like a simple framing concept, and it makes the resulting data sound charmingly like R2-D2. (It also uses begin and end markers to be double-sure of the framing.) The data is also sent with error correction, so small hiccups can get repaired automatically.

What really makes ggwave shine is that it’s ported to every platform you care about: ESP32, Arduino, Linux, Mac, Windows, Android, iOS, and anything that’ll run Python or JavaScript. So it’ll run in a browser. There’s even a GUI for playing around with alternative modulation schemes. Pshwew! This makes it easy for a minimalist microcontroller-based beeper button to control your desktop, or vice-versa. An ESP32 makes for an IoT-style WiFi-to-audio bridge. Write code on your cell phone, and you can broadcast it to any listening microcontroller. Whatever your use case, it’s probably covered.

Now the downside. The data rate is slow, around 64-160 bits per second, and the transmission is necessarily beepy-booopy, unless you pitch it up in to the ultrasound or use the radio-frequency HackRF demo. But maybe you want to hear when your devices are talking to each other? Or maybe you just think it’s cute? We do, but we wouldn’t want to have to transmit megabytes this way. But for a simple notification, a few bytes of data, a URL, or some configuration parameters, we can see this being a great software addition to any device that has a speaker and/or microphone.

Oh my god, check out this link from pre-history: a bootloader for the Arduino that runs on the line-in.

For most of us, getting weather information is as trivial as unlocking a smartphone or turning on a computer and pointing an app or browser at one’s weather site of choice. This is all well and good, but it lacks a certain panache that old weather stations had with their analog dials and stained wood cases. The weather station that [BuildComics] created marries both this antique aesthetic with modern weather data availability, and then dials it up a notch for this enormous analog weather station build.

The weather station uses 16 discrete dials, each modified with a different label for the specific type of data displayed. Some of them needed new glass, and others also needed coils to be modified to be driven with a lower current than they were designed as well, since each would be driven by one of two Arduinos in this project. Each are tied to a microcontroller output via a potentiometer which controls the needle’s position for the wildly different designs of meter. The microcontrollers themselves get weather information via the internet, which allows for about as up-to-date information about the weather as one could gather first-hand.

The amount of customization of these old meters is impressive, and what’s even more impressive is the project’s final weight. [BuildComics] reports that it took two people just to lift it onto the wall mount, which is not surprising given the amount of iron in some of these old analog meters. And, although not as common in the real world anymore, these old antique meters have plenty of repurposed uses beyond weather stations as well.

I met John Kuiphoff a couple years ago and was blown away by some very clever data visualizations he shared based on people’s youtube channels. John’s back, and this time with another very creative approach to data visualization. In this project, John is wanting to display how much sunlight hits […]

Read more on MAKE

The post These Blobs Are Actually Clever Data Visualizations appeared first on Make: DIY Projects and Ideas for Makers.

In today’s digital era, we almost take for granted that all our information is saved and backed up, be it on our local drives or in the cloud — whether automatically, manually, or via some other service.  For information from decades past, that isn’t always the case, and recovery can be a dicey process.  Despite the tricky challenges, the team at [Museo dell’Informatica Funzionante] and [mera400.pl], as well as researchers and scientists from various museums, institutions, and more all came together in the attempt to recover the Polish CROOK operating system believed to be stored on five magnetic tapes.

Originally stored at the Warsaw Museum of Technology, the tapes were ideally preserved, but — despite some preliminary test prep — the museum’s tape reader kept hanging at the 800 BPI NRZI encoded header, even though the rest of the tape was 1600 BPI phase encoding. Some head scratching later, the team decided to crack open their Qualstar 1052 tape reader and attempt to read the data directly off the circuits themselves!!

Using an Arduino Mega as a sampling device and the tape in test mode, the team were able to read the tapes, but the header remained inscrutable and accompanied by errors in the rest of the data. Promising nonetheless!

Switching gears, the decision was made to use a logic analyzer to read the tapes and use software to decode the data. While they waited for their new analyzer to ship, one of the team members, [Jacob Filipowicz] harnessed the power of Python to write a program called Nine Track Labs (pictured below) which would allow them to read any kind of magnetic tape, at any speed, BPI, and writing standard. Armed with the software and analyzer, the team was able to successfully recover the data from the tapes in its entirety without errors!

Among the data recovered, there were numerous versions of the CROOK operating system — allowing them to reproduce the OS’s development process, as well as hundreds of other files containing programs and tools hitherto believed to be lost. There was also a backup of a ‘live’ MERA-400 system with a binary CROOK-3 OS, ready to run in emulation. All things considered, the techno-archeological tour-de-force was a smashing success.

If — in your more modern travels — you need to recover an audio recording gone awry, know that you can retrieve that data with a hex editor.

Increasing accuracy in the collection of data coming from sensors is a need that, sooner or later, Makers need to face. Paul Martinsen from MegunoLink created a tutorial to eliminate noise from sensor readings on Arduino with three simple filtering techniques.

The Averaging and Running Average techniques are easy to implement as they work by adding a number of measurements together, then dividing the total by the number of measurements. In both cases, the downside is that it can use a lot of memory.

The Exponential filter is a better solution for several reasons: it doesn’t require much memory, you can control how much filtering is applied with a single parameter, and it saves battery power because you don’t need to make many measurements at once. For this solution, they developed an Arduino filter library so you don’t need to go mad with math!

Interested? You can find the tutorial and explore the code on MegunoLing’s blog post here.