The CC3000 and “always-on” sensing

Update 29/4/14

Actually, after testing for a few days, the CC3000 still becomes reliably unresponsive after ~72 hours up even after forcibly rebooting. While power-cycling it is effective to prolong run-time, afaict the firmware doesn’t seem to be up to being on for significant periods of time so buyer beware.

End Update

The CC3000 is a nifty little wifi sensor and the Adafruit shield for it is a great little sensor: small, easy to work with, fairly power conservative, fairly robust. They have a very friendly library for working with it developed by Adafruit that I’ve been using for a space sensing project that I’m doing at work and all was going smashingly, mostly, usually.

1510-00

There is a hitch though: the CC3000 can get messed up if it’s on a network that asks its clients to refresh every so often. By “messed up”, I mean not being able to connect to a network or send any data or even respond. This makes for some frustrating debugging as in my case this was happening every 8 or 9 hours or so, meaning that I’d set up my sensors, work all day, go home and look at my data feeds to find that the sensors had suddenly stopped sending data. Sometimes it was 8 hours, sometimes it was 12, sometimes it was 6. I tried just about everything available: reboot(), disconnect(), stop() in every permutation of order possible. Finally someone on the Adafruit forums helpfully pointed me to the a post on the Spark.io forum. Here’s the juicy bit:

The root cause of the CC3000 failures that lead to CFOD [error] is buffer starvation and/or allocation issues that can result in a deadlock. The situation is that the CC3000 has buffers to send, but finds it also needs buffers for either ARP packets or TCP protocol packets before it can proceed and transmit the data, but there are none available…In addition, the current behaviour of the CC3000 is to continually update its ARP cache based on packets it receives, regardless of whether those packets form part of ongoing traffic through the CC3000. With this behaviour, if the ARP cache is already full, then a random ARP cache entry is chosen and replaced. In the Core, if that ARP entry ejected is the one for the default gateway, and there are already packet(s) enqueued in the CC3000 ready to be sent, then the CC3000 must ARP to find the MAC address of the default gateway. This is apparently when the CC3000 can find itself in a deadlock, needing buffers to send the ARP request and process the reply, but not having any available. So there are a series of events, each individually contributing to the probability that the CC3000 will fail.

The Address Resolution Protocol (ARP) cache is a map on a client, like the CC3000, that associates a MAC address (an identifier for a device) with an IP address (an identifer for a device on a network). It’s pretty important for a device to have a correct ARP cache, i.e. that it knows what its IP address is on the network and be able to broadcast that out so that other things can find it and conversely it can correctly register with the network since routers like to know what’s connected to them for good reason. The CC3000 is constantly filling a limited table buffer with any ARP requests that it intercepts. This isn’t good and I’m not sure how that could be desirable behavior (I’m not a network engineer though, so I’m probably missing something important) because limited space means that at some point the ARP cache is going to fill up and the first thing in our CC3000s ARP cache, it’s own MAC address -> IP mapping, is going to get lost. No bueno. Moreover, getting a new ARP record isn’t that hard but it sounds like the CC3000 is waiting to send its packets before getting a new ARP record.

The Spark.io folks have a fix for their core that’s going to address this and at some point in the future the CC3000 core released by Texas Instruments will address this but for the moment there’s not much we direct users of the CC3000 can do except reboot the CC3000 every so often. Fortunately this is fairly easy: control the power to the CC3000 off a digital pin on the Arduino/ATmega, check to make sure it’s running properly, if it’s not, switch the pin off, count to 20 (milliseconds that is), switch it back on, start up from scratch. [I’m adding this part!] How do you turn the power on/off? Use an NPN transistor to control the power to the CC3000. I’m using the PN2222 with a 2.2k resistor on it.

Untitled Sketch_schem

The internal firmware of the CC3000 simply can’t deal with the deadlock on its own, even using its wlan_stop() and wlan_start() methods, so we give it a hand by rebooting the RTOS on the chip. My code using the Adafruit CC3000 library looks like this:

// we didn't connect at all?
  if(waitingToConnect)
  { 

    // reboot everything
    boolean disconnected = cc3000.disconnect();

    cc3000.stop();
    // reset everything in CC3000
    cc3000 = Adafruit_CC3000(ADAFRUIT_CC3000_CS, ADAFRUIT_CC3000_IRQ, ADAFRUIT_CC3000_VBAT);

    digitalWrite(WIFI_POWER_PIN, LOW);
    delay(20);
    digitalWrite(WIFI_POWER_PIN, HIGH);

    cc3000.reboot();

    // wifi
    if (!cc3000.begin()) {
      while(1);
    }

    while( cc3000.getStatus() != STATUS_CONNECTED )
    {
      // Connect to AP.
      if (!cc3000.connectToAP(WLAN_SSID, WLAN_PASS, WLAN_SECURITY)) {
        while(1);
      }

      // Wait for DHCP to be complete
      while (!cc3000.checkDHCP()) {
        delay(100);
      }
    }
  }

That’s really simplistic and there’s plenty of situations in which this could royally screw things up for, but for me, dropping a few readings while we reboot isn’t a big deal at all.

7 thoughts on “The CC3000 and “always-on” sensing

  1. Thanks for this post. Just a question. How is the waitingToConnect variable set? That part seems to be missing in your code. Where it’s is becoming true, so the reboot is initiated?

  2. Hi, Do you know how much current a CC3000 draws at peak compared to what an Arduino Uno can supply through a digital out pin ? I thought it was much more that recommended for Arduino supply over a digital pin, but it ,must be working for you :)

  3. Hey, sorry, I wrote this up real quick on a flight and forgot to mention that I’m routing it through an NPN transistor + resistor on the Arduino. Just corrected the post to mention that and will add a schematic when I get some time :)

  4. joshuajnoble,

    What is the smaller package –is it the cc3000? The labeling text is not readable on my monitor…

    Thanks for the Fritzing diagram

    Wm.

  5. It is the cc3000 though the label says TRF24G for some reason (probably being in a rush and dropping the first Fritzing object with the correct # of pins on it into the doc).

  6. Thank you for this information. I’ve experienced similar issues with the breakout boards and been unable to resolve them. Inevitably, the device works for a while and drops off the network. I don’t see any discussion on the sites using the chip in their breakout boards that warn that its not appropriate for long-running use. I was attempting to run a rest service on it and integrate it with OpenRemote to detect state. It was polled every 5 seconds- and works fine for the first 6 to 8 hours or so. I think I’m giving up on it in my project.

  7. Hi,
    The connection to the NPN transistor seems a bit odd to me, why did you connect the load to the emitter and the GND to the collector?
    Doesn’t the Vce voltage drop causes issues in the CC3000?
    Can it be connected so the load is on the collector and GND is on the emitter?
    Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *