UDP (User Datagram) lacks some of the nice features of TCP (no gaurantees with regard to packet delivery, and no guarantee that packets will be delivered in the correct order), but it also lacks much of the overhead associated with TCP. This makes it fast, and it also makes it a very nice starting point for working with LWIP! This webpage shows how to setup a simple chat capability between a PC and and Pi Pico W.
The code that this webpage documents lives here.
Some understanding of the data structures that LWIP uses to manage network connections and for communicating network data through the stack is helpful for assembling and debugging these programs. There are two data structures in particular that are useful to understand: protocol control blocks (PCB's) and packet buffers (pbuf's). Let's consider each in turn.
Your application code may be managing multiple network connections at the same time, perhaps receiving information over one port from one remote IP address and sending information out to some other destination address over a different port. LWIP uses a protocol control block (PCB) to manage each of these network connections. Each PCB is a struct that contains all of the information of relevance for a particular connection. In the case of a UDP protocol control block, this information includes things like the local IP, remote IP, local and remote ports, and a pointer to a receive callback function (plus some flags for settings).
Perhaps it stands to reason that the protocol control block for a TCP connection maintains a lot more state than that for UDP. UDP is connectionless and therefore receives no receipt of packet acknowledgements. It has no error correction (beyond a checksum in each packet) and therefore maintains no state associated with retransmission attempts.
Packet buffers (PBUF's) are used to shuffle data through the LWIP stack. These are the containers that hold the packet data, and some metainformation about that packet. Each contains a pointer to the payload (the actual data), a field that indicates the total length of the buffer, a field that contains the length of this particular buffer, and a pointer to another PBUF called next. In the event that the packet data exceeds that which can be held in a single buffer, this pointer allows for us to chain PBUF's in a linked list. Other fields include the PBUF type (does this PBUF live in RAM, ROM, etc.?), some flags, and a reference count. The reference count tracks the number of users of this PBUF so that it's not freed to soon (more on that later). The final fields include one to hold the netif index for an incoming packet, and a place where a user could store custom data on a PBUF.
This is how LWIP organizes packet information! When we want to send a packet, we will allocate a PBUF, populate its payload field, and send it out by way of a protocol control block. When we receive a packet, LWIP will emit that packet to our callback function as a PBUF. As we'll see, it's our responsibility to free each PBUF once we've finished using it. We interact with these PBUFF's largely by way of the PBUF API.
Setting up our device to receive UDP packets involves the following steps:
Let us examine each of these steps.
We can declare a UDP protocol control block globally, and then we'll assign its value in an initialization function. Declaring the PCB (or, more specifically, a pointer to the PCB) looks like the following:
// Protocol control block for UDP receive connection
static struct udp_pcb *udp_rx_pcb;
And the initializing this pcb looks like this:
// Initialize the RX protocol control block
udp_rx_pcb = udp_new_ip_type(IPADDR_TYPE_ANY);
Under the hood, udp_new_ip_type makes a call to the function udp_new(), which allocates a new udp protocol control block by way of (struct udp_pcb *)memp_malloc(MEMP_UDP_PCB);.
It may seem unusual that we would only declare a pointer to a PCB without allocating any memory, at compile-time, for the PCB itself. It may indeed make you very uncomfortable that we are doing what looks like a malloc in an embedded program. Might this not lead to memory fragmentation?
LWIP does not use standard malloc/free in order to allocate and free protocol control blocks or PBUF's. Instead, in the case of PCB's, it statically allocates a pool of protocol control block objects. When we runtime-allocate a new udp_pcb, LWIP is actually just grabbing one of the PCB's from this pool. When we free a PCB, LWIP is actually just marking it as "not in use." So, in the case of PCB's, there is not any dynamic memory allocation occuring at runtime, though the LWIP API vocabulary kind of makes it sound like there is. The same applies to the PBUF structures (though not, as we'll see, to the PBUF payloads). The number of PBUF's in the pool is configurable by way of an LWIP configuration, PBUF_POOL_SIZE. And the number of UDP PCB's in the pool is another LWIP configuration, MEMP_NUM_UDB_PCB.
For PBUF payloads, LWIP does in fact dynamically allocate memory from heap. But it does so from its own dedicated memory, of a size that we specify in the configuration, and by way of its own implementation of malloc that prohibits allocation outside of this dedicated memory. Even so, memory fragmentation is a risk! If you application includes lots of allocating/freeing of different-sized PBUF's, you may encounter this problem. You can mitigate your risk by instead allocating PBUF's from the PBUF pool. This pool contains pre-allocated memory for use as PBUF payloads. Using the pool means that your payload size is always the same (though you get to specify what that size is, by way of an LWIP configuration), which means you'll likely be using more memory than you would by allocating. But! You also aren't doing any dynamic memory allocation, which mitigates the risk of fragmentation. Note that all PBUF's allocated for received UDP packets come from this pool.
When we bind a PCB, we are associating it with a particular local IP and port. When a packet appears at that IP and port, it will be directed to our particular PCB (with which we can then associate a receive callback function). The bind occurs in the following line:
err = udp_bind(udp_rx_pcb, netif_ip_addr4(netif_default), UDP_PORT);
The first argument is the PCB that we've just created, the second is the default IP address that we've been assigned by DHCP (see here for details), and the final argument is the port number. Under the hood, this function is setting the values of the local_port and local_ip fields of the udp_pcb struct. This function will return an error code which will either indicate that binding was successful (ERR_OK), or will indicate that something went wrong. Perhaps we attempted to bind a port which was already bound to another PCB, or perhaps we specified an invalied IP or port, or perhaps there was a memory allocation issue. Before proceeding, we check this error code to confirm that everything was successful.
Having associated a local IP and port with our PCB, we'd now like to setup a receive callback function. When a packet appears at our bound IP and port address, it will be directed to this callback function. LWIP allows for us to do this by way of the udp_recv function:
if (err == ERR_OK) {
// Setup the receive callback function
udp_recv(udp_rx_pcb, udpecho_raw_recv, NULL);
} else {
printf("bind error");
}
The first argument is the PCB that we've just created and bound, the second argument is a function pointer to the callback function, and the third argument allows for us to pass some user-defined arguments to this callback functions. In the above case, we aren't passing any arguments to the callback function. Having set this up, any packet that appears to the udp_rx_pcb PCB, which is bound to our specified local IP and port, will be forwarded to our specified callback function. Let's now take a look at that function.
LWIP will call our callback function with a standard set of arguments, which include the (optional) user specified argument pointer, a pointer to the PCB, a pointer to a PBUF which holds the received packet, a pointer to the IP address object associated with the packet, and the port number to which the packet appeared.
Within the callback function, we copy the payload of the packet to a buffer, signal a semaphore for a thread to run, clear the payload buffer, and then free the PBUF. It is our responsibility to free this PBUF.
static void udpecho_raw_recv(void *arg, struct udp_pcb *upcb, struct pbuf *p,
const ip_addr_t *addr, u16_t port)
{
LWIP_UNUSED_ARG(arg);
// Check that there's something in the pbuf
if (p != NULL) {
// Copy the contents of the payload
memcpy(received_data, p->payload, BEACON_MSG_LEN_MAX) ;
// Semaphore-signal a thread
PT_SEM_SAFE_SIGNAL(pt, &new_message) ;
// Reset the payload buffer
memset(p->payload, 0, BEACON_MSG_LEN_MAX+1);
// Free the PBUF
pbuf_free(p);
}
else printf("NULL pt in callback");
}
The associated thread which runs in response to this semaphore from the callback function simply prints out the message that we received.
static PT_THREAD (protothread_receive(struct pt *pt))
{
// Begin thread
PT_BEGIN(pt) ;
while(1) {
// Wait on a semaphore
PT_SEM_SAFE_WAIT(pt, &new_message) ;
// Print received message
printf("%s\n", received_data);
}
// End thread
PT_END(pt) ;
}
Setting up our device to send UDP packets involves the following steps:
Let us take a look at each of these steps.
All of our UDP send logic is consolidated in a single Protothread. In the initialization part of that thread, we have the two lines of code indicated below. The first of these creates a static local pointer to an object of type udp_pcb called pcb, and the second allocated the udp_pcb object to which this pointer points. This is a static pointer so that it retains its value even after this thread yields.
// Make a static local UDP protocol control block
static struct udp_pcb* pcb;
// Initialize that protocol control block
pcb = udp_new() ;
It's handy to create a static local object of type ip_addr_t to hold the destination IP address. The two lines below create such an object, and then populate the value of that object with the destination IP address.
// Create a static local IP_ADDR_T object
static ip_addr_t addr;
// Set the value of this IP address object to our destination IP address
ipaddr_aton(BEACON_TARGET, &addr);
In this particular example, we will send user-provided character strings. This thread prompts the user to enter a string via a serial commad line. This string gets stored in the character array pt_serial_in_buffer.
// Prompt the user
sprintf(pt_serial_out_buffer, "> ");
serial_write ;
// Perform a non-blocking serial read for a string
serial_read ;
Having gathered the information that we'd like to transmit, we must next allocate a PBUF in which to store that information. The line below creates a pointer to a pbuf object, and allocates that PBUF object to which the pointer points.
// Allocate a PBUF
struct pbuf *p = pbuf_alloc(PBUF_TRANSPORT, BEACON_MSG_LEN_MAX+1, PBUF_RAM);
The first of these arguments indicates the protocol layer for which the pbuf is being allocated. By specifying this as being on the PBUF_TRANSPORT layer, we are telling LWIP that this PBUF is intended for UDP communication and it will automatically allocate space for necessary packet headers in addition to the data. Alternatives for this argument inclue PBUF_IP, which would indicate that this is a network-layer PBUF, or PBUF_LINK (link layer), or PBUF_RAW (no headers).
The second argument indicates the size of the payload that should be allocated (in bytes). We've added a "+1" to account for null-terminated C strings.
The final argument tells LWIP the place from which it should grab or allocate the PCB. By making this argument PBUF_RAM, we are specifying that this PBUF should be allocated from RAM. We could instead have made this argument PBUF_POOL, in which case the PBUF would be grabbed from a compile-time-allocated pool of PBUF's (all of fixed payload size). We could alternatively have specified PBUF_ROM, which references read-only memory. Finally, this argument could be PBUF_REF to indicate external memory.
We next populate the payload field of the PBUF that we've just allocated. As a matter of convenience, we first create a pointer to a char, and assign its value to be the address of the payload field of the PBUF. Now, be dereferencing and assigning values to this character pointer, we are setting the payload of the PBUF.
We next clear the payload of the PBUF, and then use snprintf and some string formatters to write BEACON_MSG_LEN_MAX+1 bytes to the payload. The payload will include our IP address, followed by the contents of pt_serial_in_buffer.
// Pointer to the payload of the pbuf
char *req = (char *)p->payload;
// Clear the pbuf payload
memset(req, 0, BEACON_MSG_LEN_MAX+1);
// Fill the pbuf payload
snprintf(req, BEACON_MSG_LEN_MAX, "%s: %s \n",
ip4addr_ntoa(netif_ip_addr4(netif_default)), pt_serial_in_buffer);
Having allocated and populated our PBUF, we next want to send it! We do this by way of the udp_sendto function shown below. The first argument to this function is the PCB which we've created to manage our UDP transmissions, the second is a pointer to our PBUF, the third is a pointer to the ip_addr_t object which holds the destination IP, and the final argument specifies the destination port address.
Note that we have not bound this PCB to any local port! As a consequence, LWIP will assign a random source port to the packet. We could bind this to a local port, if we cared about the source port. Note also that we have a few options with regard to sending a udp packet. Rather than using udp_sendto, which takes the destination IP/port as arguments, we could alternatively have used udp_connect to associated fixed destination IP/port values with this PCB and then sent the packet using udp_send.
// Send the UDP packet
err_t er = udp_sendto(pcb, p, &addr, UDP_PORT);
Now that we're finished with it, free it! In the event that this PBUF was allocated from RAM, this will free the associated memory. In the event that we grabbed this PBUF from the pool, this will mark it as available.
// Free the PBUF
pbuf_free(p);
Listed below.
static PT_THREAD (protothread_send(struct pt *pt))
{
// Begin thread
PT_BEGIN(pt) ;
// Make a static local UDP protocol control block
static struct udp_pcb* pcb;
// Initialize that protocol control block
pcb = udp_new() ;
// Create a static local IP_ADDR_T object
static ip_addr_t addr;
// Set the value of this IP address object to our destination IP address
ipaddr_aton(BEACON_TARGET, &addr);
while(1) {
// Prompt the user
sprintf(pt_serial_out_buffer, "> ");
serial_write ;
// Perform a non-blocking serial read for a string
serial_read ;
// Allocate a PBUF
struct pbuf *p = pbuf_alloc(PBUF_TRANSPORT, BEACON_MSG_LEN_MAX+1, PBUF_RAM);
// Pointer to the payload of the pbuf
char *req = (char *)p->payload;
// Clear the pbuf payload
memset(req, 0, BEACON_MSG_LEN_MAX+1);
// Fill the pbuf payload
snprintf(req, BEACON_MSG_LEN_MAX, "%s: %s \n",
ip4addr_ntoa(netif_ip_addr4(netif_default)), pt_serial_in_buffer);
// Send the UDP packet
err_t er = udp_sendto(pcb, p, &addr, UDP_PORT);
// Free the PBUF
pbuf_free(p);
// Check for errors
if (er != ERR_OK) {
printf("Failed to send UDP packet! error=%d", er);
}
}
// End thread
PT_END(pt) ;
}
In my view, the easiest way to communicate with the Pico W from a PC is with netcat. This is a linux networking utility.
To send strings to the Pico W, I open a terminal window and run the command shown below. This is a netcat command (nc). The -u flag indicates UDP transactions, 172.20.10.5 is the destination IP (that of the Pico), and 1234 is the destination port (the one to which we've bound our receive PCB on the Pico). Once I run this, I can enter strings and, each time I push the enter key, that string will be transmitted, it will be forwarded to our receive PCB, which will call our callback function, which will copy the payload and semaphore-signal the receive thread, which will print out our message.
nc -u 172.20.10.5 1234
To receive messages on our PC sent to us by the Pico, we run the command below in a separate terminal window. This is a netcat command which says please listen (-l) for udp packets (-u) on port 1234. This is the same port that we've specified as the destination port in the udp_sendto command described above. If we run this, then each time the Pico sends us a message it will be printed to the terminal window.
nc -u -l 1234
Perhaps you're building some sort of Python application on the PC, and you'd like for that application to communicate with your Pico. The Python program below sets up a simple chat interface that allows for you to send and receive character strings.
To make this code talk to your Pico, I recommend the following:
en0 to see what computer's IP is. Mine is 172.20.10.2import socket
import threading
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind(('172.20.10.2', 1234)) # (host, port) because AF_INET
print("Listening...")
def UDP_rx():
while True:
print(sock.recv(100).decode()) # buffer size
t1 = threading.Thread(target=UDP_rx, args=())
t1.start()
while True:
message = input("Send message: ")
sock.sendto(bytes(str(message), "utf-8"), ("172.20.10.5", 1234))
message = ""