Arduino Ethernet parsing XML - http

I'm using an Arduino Uno and ethernet shield to pull xml data from OpenWeatherMap.org. I have the api link and a key, and it all works fine on the web browser. My code is below, however when I get to reading the response from the GET command, all i get is "ΓΏ", not anything I would expect.
I can connect to the arduino when using it as a server, and I can ping it from command prompt so I know it is on the network and the board is not at fault, so it must be something with my code. I have followed the tutorial on the arduino site and loaded the example to GET data, but this also does not work, all I get is a "did not connect" message.
Can someone please look over my code and see what I am doing wrong?
EDIT: I have added the extra loop to print all the xml data, however the program still gets stuck on the while(!client.available());. If i comment it out, I get to the "waiting for server response" but never any further than that. I have checked that the arduino si on the same subnet mask as all the other devices in the network.
// Based on:
// Read Yahoo Weather API XML
// 03.09.2012
// http://forum.arduino.cc/index.php?topic=121992.0
//
#include <SPI.h>
#include <Ethernet.h>
#include <TextFinder.h>
int cityID=2644487; //Lincoln, UK
byte mac[] = {0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x02};
byte ip[] = {192, 168, 1, 89};
byte gateway[] = {192, 168, 1, 254};
byte subnet[] = {255, 255, 255, 0};
//Open weather map xml
char server[] = "http://api.openweathermap.org";
int port = 80; //usually 80 for http.
char APIkey[33] = "HIDDENAPIKEY";
EthernetClient client;
char temperature[30];
void setup()
{
pinMode(10, OUTPUT);
digitalWrite(10,HIGH);
Serial.begin(9600);
Serial.println("Initialising...");
// Start Ethernet
if(!Ethernet.begin(mac)){//if DHCP does not automatically connect
Serial.println("Invalid Connection");
}
Serial.println("");
Serial.print("Connecting to OWM server using cityID: ");
Serial.println(cityID);
Serial.print("Using API key: ");
Serial.println(APIkey);
if (client.connect(server,port))
{
client.println("GET /data/2.5/weather?id=2644487&appid=HIDDENAPIKEY&mode=xml&units=metric HTTP/1.1");
client.println("HOST: api.openweathermap.org");
client.println();
Serial.println("Connected to XML data.");
while(!client.available()); //wait for client data to be available
Serial.println("Waiting for server response...");
while(client.available()){
char c = client.read();
Serial.println(c);
}
}
}
void loop()
{
}

For start, your Host header is wrong, use only domain name:
client.println("HOST: api.openweathermap.org");
Second problem I see here is that you do not read entire response from server but only one char.
You have to wait for response data to be available, after that read all the data and parse it or print at start.
This part is wrong:
char c = client.read();
Serial.println(c);
Should be something similar to:
while (!client.available()); //Wait client data to be available
while (client.available()) { //Print all the data
char c = client.read();
Serial.println(c);
}

Related

Unable to connect to MQTT Server

I am using knolleary library to connect the Arduino UNO board to MQTT server. For broker I am using test.mosquitto.org (85.119.83.194) but I am not able to connect.
Here is my code:
/*
Basic MQTT example
- connects to an MQTT server
- publishes "hello world" to the topic "outTopic"
- subscribes to the topic "inTopic"
*/
#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
// Update these with values suitable for your network.
byte mac[] = { 0xDE, 0xED, 0xBA, 0xFE, 0xFE, 0xED };
byte server[] = { 10, 2, 63, 123};
byte ip[] = { 85, 119, 83, 194 };
void callback(char* topic, byte* payload, unsigned int length) {
// handle message arrived
Serial.println("Message received");
}
EthernetClient ethClient;
PubSubClient client(server, 1883, callback, ethClient);
void setup()
{
Ethernet.begin(mac, ip);
Serial.begin(9600);
Serial.println("Ethernet Begin");
if (client.connect("arduinoClient")) {
Serial.println("Client connected");
client.subscribe("/notification/turnlighton");
}
else{
Serial.println("Client not connected");
}
}
void loop()
{
client.loop();
}
client.connect("arduinoClient") return false and "Client not connected" message is printed Serial Monitor.
I am not sure what should be the value of
byte server[] = { 10, 2, 63, 123};
As an alternate I also tried to connect to the Really Simple Message Broker (RSMB) in intranet. Still I get same message.
Can any one help in here?
Thanks in advance
SRS
You have it around the wrong way;
byte server[] is your mqtt server's ip address, in your case test.mosquitto.org (85.119.83.194)
byte ip[] is the static ip address you want the arduino to have on your network.
The other thing to check is that you can connect using a cli client to test.mosquitto.org as it is sometimes down I find.
Have a look at my temperature publishing code, https://github.com/matbor/arduinoTemps2mqtt it might give you a few hints as it was modified from that original example that comes with PubSubClient. In my one I just have the arduino running off DHCP.

Arduino Knolleary PubSubClient will publish messages but can't receive them

I am using the Knolleary PubSubClient to make a connection to my MQTT server. I have been able to successfully authenticate and make a connection after not much work. I can even publish messages to topics. However, the issue I am having is that I can subscribe to topics and get no error, but when I publish to that topic (from mosquitto on my Mac) the callback does not get called and the message to the subscribe topic does not appear to be received. I have tried running a mosquitto subscription to the same topic at the same time, and that does receive the published message. Not sure if there is a problem in my callback code or what is going on here. Any help would be appreciated. Arduino code is below:
/*
Basic MQTT example
- connects to an MQTT server
- publishes "hello world" to the topic "outTopic"
- subscribes to the topic "inTopic"
*/
#include <SPI.h>
#include <Ethernet.h>
#include <PubSubClient.h>
// Update these with values suitable for your network.
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte server[] = { 10, 2, 63, 123 };
byte ip[] = { 192, 168, 1, 10 };
void callback(char* topic, byte* payload, unsigned int length) {
Serial.println(topic);
//convert byte to char
payload[length] = '\0';
String strPayload = String((char*)payload);
Serial.println(strPayload);
//int valoc = strPayload.lastIndexOf(',');
//String val = strPayload.substring(valoc+1);
//Serial.println(val);
}
EthernetClient ethClient;
PubSubClient client(server, 1883, callback, ethClient);
void setup()
{
Serial.begin(19200);
Serial.println("==STARTING==");
// start the Ethernet connection:
if (Ethernet.begin(mac) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
// no point in carrying on, so do nothing forevermore:
// try to congifure using IP address instead of DHCP:
Ethernet.begin(mac, ip);
}
// give the Ethernet shield a second to initialize:
delay(1000);
Serial.println("connecting...");
for (byte thisByte = 0; thisByte < 4; thisByte++) {
// print the value of each byte of the IP address:
Serial.print(Ethernet.localIP()[thisByte], DEC);
Serial.print(".");
}
//delay(500);
boolean con = client.connect("1", "3snzzon5dyade:abc", "OBSCURED_FOR_SEC");
while(con != 1){
Serial.println("no con-while");
con = client.connect("1", "3snzzon5dyade:abc", "OBSCURED_FOR_SEC");
}
//Serial.println(con);
if(con){
Serial.println("got con");
client.publish("testq","hello world");
client.subscribe("testq");
}else Serial.println("no con");
}
void loop()
{
client.loop();
}
Like I said, I can see everything working properly with mosquitto. I have even tried matching up client_ids with no luck. Any help or ideas would be greatly appreciated.
your subscribe topic "testq" must be in an array like
char testq[] = {'t', 'e', 's', 't', 'q', '\0'};
be sure to finisht your array with an ,'\0'
Then you subscribe with:
client.subscribe(testq);

Arduino Ethernet Reading from PHP File

I am writing a simple Arduino Ethernet program. The program sends an HTTP GET Request to the server and then the server echos "Hello World" and I should be able to receive it through the Arduino Ethernet and print it on the Serial monitor of the Arduino 1.0.4 IDE. Here are some useful information. I am using a XAMPP Server on Windows Server 2003. I have placed my PHP file in /xampp/htdocs/xampp and the file name is rec.php. The contents of rec.php is
<?php
echo "Hello World";
?>
This is the file content of the Arduino program
#include <Ethernet.h>
#include <SPI.h>
byte mac[] = {0x90, 0xA2, 0xDA, 0x00, 0x7E, 0xAE}
IPAddress server { 192, 168, 1, 223 };
IPAddress ipAddress { xxx,xxx,xxx,xxx };
IPAddress myDNS {8,8,8,8};
IPAddress myGateway{192,168,1,1};
IPAddress mySubnet{255,255,255,0};
EthernetClient client;
void setup()
{
Serial.begin(9600);
Ethernet.begin(mac, ipAddress, myDNS, myGateway, mySubnet);
delay(1000);
Serial.println("connecting");
if(client.connect(server, 80))
{
Serial.println("Connected");
client.println("GET /rec.php HTTP/1.1");
}
else
Serial.println("Not Connected");
}
void loop()
{
if(client.available())
{
char c = client.read();
Serial.println(c);
delay(1000);
}
else
{
Serial.println("Not Available");
delay(1000);
}
}
After I load the program on the Arduino I get this message on the Serial Monitor "HTTP/1.1 400 Bad Request". Any suggestion on how to solve that problem? and please keep your answers simple.
You are not sending the necessary line ends. The protocol requires a CR-LF at the end of the request method and another CR-LF at the end of the full request that can include other header lines. See:
HTTP requests
This means in your case you need two CR-LF's to terminate the request. Don't rely on default println function. Take control of the line ends in your code with print:
client.print("GET /rec.php HTTP/1.1\r\n\r\n");

Arduino DHCP error

I'm testing the Ethernet shield with an Arduino Uno, and I'm getting a DHCP error just using the example sketch.
#include <SPI.h>
#include <Ethernet.h>
byte MACaddress[] = { 0x90, 0xAD, 0xDA, 0x0D, 0x96, 0xFE };
EthernetClient client;
void setup() {
Serial.begin(9600);
while (!Serial) {
;
}
// Start the Ethernet connection:
if (Ethernet.begin(MACaddress) == 0) {
Serial.println("Failed to configure Ethernet using DHCP");
for(;;)
;
}
Serial.print("My IP address: ");
for (byte thisByte = 0; thisByte < 4; thisByte++) {
Serial.print(Ethernet.localIP()[thisByte], DEC);
Serial.print(".");
}
Serial.println();
}
void loop() {
}
I've opened the router administration page, and I can see it gave the Arduino an IP address, associated with the MAC address. I've also tried a static IP address in the code (Ethernet.begin(MACaddress, IPaddress)), but it won't work either.
I can't ping the shield IP address that shows in the router administrator page.
What is wrong with just this simple code?
Everything is out of the box, the Arduino and the shield. I haven't done anything with them, just connected the shield to the Arduino and sent the code. It seems everything is working fine, the LEDs are blinking for both boards.
These loops are useless.. Could you try something like:
#if defined(ARDUINO) && ARDUINO > 18
#include <SPI.h>
#endif
#include <Ethernet.h>
#include <EthernetDHCP.h>
// MAC Address
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
const char* ip_to_str(const uint8_t*);
// Initialize the Ethernet server library
Server server(8080);
void setup()
{
Serial.begin(9600);
Serial.println("Attempting to obtain a DHCP lease...");
// Initiate a DHCP session. The argument is the MAC (hardware) address that
// you want your Ethernet shield to use. This call will block until a DHCP
// lease has been obtained. The request will be periodically resent until
// a lease is granted, but if there is no DHCP server on the network or if
// the server fails to respond, this call will block forever.
// Thus, you can alternatively use polling mode to check whether a DHCP
// lease has been obtained, so that you can react if the server does not
// respond (see the PollingDHCP example).
EthernetDHCP.begin(mac);
// Since we're here, it means that we now have a DHCP lease, so we print
// out some information.
const byte* ipAddr = EthernetDHCP.ipAddress();
const byte* gatewayAddr = EthernetDHCP.gatewayIpAddress();
const byte* dnsAddr = EthernetDHCP.dnsIpAddress();
Serial.println("A DHCP lease has been obtained.");
Serial.print("My IP address is ");
Serial.println(ip_to_str(ipAddr));
Serial.print("Gateway IP address is ");
Serial.println(ip_to_str(gatewayAddr));
Serial.print("DNS IP address is ");
Serial.println(ip_to_str(dnsAddr));
// Start the server
server.begin();
}
void loop()
{
// You should periodically call this method in your loop(): It will allow
// the DHCP library to maintain your DHCP lease, which means that it will
// periodically renew the lease and rebind if the lease cannot be renewed.
// Thus, unless you call this somewhere in your loop, your DHCP lease might
// expire, which you probably do not want :-)
EthernetDHCP.maintain();
// listen for incoming clients
Client client = server.available();
if (client) {
// an http request ends with a blank line
boolean currentLineIsBlank = true;
while (client.connected()) {
if (client.available()) {
char c = client.read();
// if you've gotten to the end of the line (received a newline
// character) and the line is blank, the http request has ended,
// so you can send a reply
if (c == '\n' && currentLineIsBlank) {
// send a standard http response header
client.println("HTTP/1.1 200 OK");
client.println("Content-Type: text/html");
client.println();
// Some misc. HTML
client.println("<title>Arduino Control Panel</title>");
client.println("<center><h1>Control Panel</h1></center>");
client.println("<p></p>");
// output the value of each analog input pin
for (int analogChannel = 0; analogChannel < 6; analogChannel++) {
client.print("Analog input ");
client.print(analogChannel);
client.print(" is ");
client.print(analogRead(analogChannel));
client.println("<br />");
}
break;
}
if (c == '\n') {
// you're starting a new line
currentLineIsBlank = true;
}
else if (c != '\r') {
// you've gotten a character on the current line
currentLineIsBlank = false;
}
}
}
// give the web browser time to receive the data
delay(1);
// close the connection:
client.stop();
}
}
// Just a utility function to nicely format an IP address.
const char* ip_to_str(const uint8_t* ipAddr)
{
static char buf[16];
sprintf(buf, "%d.%d.%d.%d\0", ipAddr[0], ipAddr[1], ipAddr[2], ipAddr[3]);
return buf;
}
I'm not sure what you mean by "I've also tried a static IP address in the code". If you simply replaced
if (Ethernet.begin(MACaddress) == 0) {
with
if (Ethernet.begin(MACaddress, myIP) == 0) {
the result may be unpredictable, because there is no return value.
Read
EthernetBegin
Returns
The DHCP version of this function, Ethernet.begin(mac), returns an int: 1 on a successful DHCP connection, 0 on failure. The other versions don't return anything.
have you tried one of the examples with fixed IP's?

My Arduino + Ethernet shield WebServer sketch sometimes fails to connect to the client. Causes?

My Arduino web server sketch sporadically fails on:
EthernetClient client = server.available();
if (client)
This morning, it connected just fine on the first run. Now, it can't connect to the client again. A couple of days ago, it worked several times, but failed several time as well. I have the shield connected via an Ethernet cable to my home router. I've verified the IP address assigned to the Arduino. I've tried ports 80 and 8080. What could be going wrong and what else can I try? Could my ISP be blocking something here? Please don't be afraid to suggest the obvious, since I know almost nothing about networks.
If relevant, here is a larger piece of the code, which loops on
Serial.println("Listening");
Code:
#include <SPI.h>
#include <Ethernet.h>n
// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = { 0x90, 0xA2, 0xDA, 0x00, 0xF7, 0x99 };
IPAddress ip(192,168,2,5);
// Initialize the Ethernet server library
// with the IP address and port you want to use
// (port 80 is default for HTTP):
EthernetServer server(80);
String roundOpenTag = "";
String roundCloseTag = "";
void setup()
{
// Start the Ethernet connection and the server:
Ethernet.begin(mac, ip);
server.begin();
int ledPin = 8;
// Initialize the digital pin as an output.
pinMode(ledPin, OUTPUT);
Serial.begin(9600);
Serial.println("Setting up");
}
void loop()
{
// Listen for incoming clients
EthernetClient client = server.available();
Serial.println("Listening");
if (client)
{
Serial.println("Server available");
// An HTTP request ends with a blank line
boolean currentLineIsBlank = true;
while (client.connected()) {
Serial.println("Client connected");
if (client.available())
{
char c = client.read();
I don't see the purpose in including the rest of the sketch. I really appreciate your help.
You have a empty Seiral.begin() in your setup() function. Try removing it.
Edit:
When you call Serial.begin() you have to provide the baut rate(speed) at which you want to communicate. You can read more about the function at Arduino library page.
You had two problems in your code
You had a empty Serial.begin() function call, without any parameter
You had duplicate Serial.begin() function. You had already specified it in the beginning of the setup() function.

Resources