1. Brian Thorne
  2. python-socket-examples


python-socket-examples / presentation.html

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta http-equiv="Content-Style-Type" content="text/css" />
  <meta name="generator" content="pandoc" />
  <meta name="author" content="Brian Thorne @thorneynz" />
  <style type="text/css">
table.sourceCode, tr.sourceCode, td.lineNumbers, td.sourceCode {
  margin: 0; padding: 0; vertical-align: baseline; border: none; }
table.sourceCode { width: 100%; }
td.lineNumbers { text-align: right; padding-right: 4px; padding-left: 4px; color: #aaaaaa; border-right: 1px solid #aaaaaa; }
td.sourceCode { padding-left: 5px; }
code > span.kw { color: #007020; font-weight: bold; }
code > span.dt { color: #902000; }
code > span.dv { color: #40a070; }
code > span.bn { color: #40a070; }
code > span.fl { color: #40a070; }
code > span.ch { color: #4070a0; }
code > span.st { color: #4070a0; }
code > span.co { color: #60a0b0; font-style: italic; }
code > span.ot { color: #007020; }
code > span.al { color: #ff0000; font-weight: bold; }
code > span.fu { color: #06287e; }
code > span.er { color: #ff0000; font-weight: bold; }
  <link rel="stylesheet" type="text/css" media="screen, projection, print"
    href="http://www.w3.org/Talks/Tools/Slidy2/styles/slidy.css" />
  <script src="http://www.w3.org/Talks/Tools/Slidy2/scripts/slidy.js.gz"
    charset="utf-8" type="text/javascript"></script>
<div class="slide titlepage">
  <h1 class="title">Sockets?</h1>
  <p class="author">
Brian Thorne @thorneynz
  <p class="date">Kiwi Python Conference 2012</p>
<div class="section slide level1" id="section">
<h1 id="section"></h1>
<!-- pandoc -s --highlight-style=pygments --webtex -i -t slidy slideshow.md -o presentation.html -->

<div class="section slide level1" id="seriously-a-talk-on-sockets">
<h1 id="seriously-a-talk-on-sockets">Seriously a talk on Sockets?</h1>
<ul class="incremental">
<li>Who the hell would give a talk on sockets?</li>
<li>This talk isn't really about sockets</li>
<li>(well not entirely)</li>
<li>now that is cleared up lets get <strong>sockets</strong> out of the way...</li>
<div class="section slide level1" id="network-sockets">
<h1 id="network-sockets">Network Sockets</h1>
<p>The rather normal network socket:</p>
<pre><code>import socket
tcp_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
tcp_socket.connect((&quot;google.com&quot;, 80))
tcp_socket.sendall(b&quot;GET / HTTP/1.1\r\n&quot;)</code></pre>
<ul class="incremental">
<li>A not so distant cousin to how your browser kicks off a request...</li>
<li>Let's take a look at the normal client/server example for network sockets</li>
<div class="section slide level1" id="server">
<h1 id="server">Server</h1>
<pre><code>def echo(conn):
    while True:
        data = conn.recv(1024)
        if not data: break

def consumer_server(...
    s = socket.socket(address_family, socket_type, protocol)
    while True:
        conn, addr = s.accept()
        threading.Thread(target=echo, args=(conn,)).start()</code></pre>
<ul class="incremental">
<li>listen(5) tells the OS to queue up to 5 connection requests</li>
<li>Note a new thread gets spun up for every client connection</li>
<li>Threads spin up plenty quick</li>
<li>Yes all the error handling has been removed to ease readability at the expense of all usability..</li>
<div class="section slide level1" id="client">
<h1 id="client">Client</h1>
<pre><code>af, socktype, proto, _, sa = socket.getaddrinfo(
s = socket.socket(af, socktype, proto)
<ul class="incremental">
<li>Leaving the address family unspecified will return INET and INET6</li>
<li>This would be a bit of a boring demo (at least running on my laptop)</li>
<div class="section slide level1" id="neo-freerunner">
<h1 id="neo-freerunner">Neo Freerunner</h1>
<p>Meet the Freerunner - open source phone running SHR (debian based)</p>
<pre><code>ssh root@

cd code
cat server.py
python server.py</code></pre>
<ul class="incremental">
<li><p>On the other end (e.g. my pc) run client(s)</p>
<p>python3 basic_socket_client.py</p></li>
<div class="figure">
<img src="images/freerunner.jpg" /><p class="caption"></p>
<div class="section slide level1" id="unix-sockets">
<h1 id="unix-sockets">Unix Sockets</h1>
<p>So on GNU/Linux <em>everything</em> is a file. This includes everything from .txt files to Sockets.</p>
<ul class="incremental">
<li><p>TCP/UDP sockets <strong>can</strong> connect endpoints on the same network</p></li>
<li><p>Unix Sockets can only connect endpoints on the same machine.</p></li>
<li><p>This lets the Kernel make some assumptions and cut out the<br />expensive &quot;networking&quot; layer.</p></li>
<div class="section slide level1" id="unix-sockets-1">
<h1 id="unix-sockets-1">Unix Sockets</h1>
<p>They are extensively used for Inter Process Communicaton, including internally in Python's multiprocessing.connection module:</p>
<pre><code># A higher level module for using sockets (or Windows named pipes)
# multiprocessing/connection.py</code></pre>
<ul class="incremental">
<li><p>From Python 3.3 Unix Sockets can also be used to send open file descriptors (open files or sockets) to another Process using the sendmsg() and recvmsg() calls.</p></li>
<li><p>If you require interprocess communication in Python multiprocessing.Queue is a much better and easier way!</p></li>
<div class="section slide level1" id="socket-abuse">
<h1 id="socket-abuse">Socket (ab)use</h1>
<ul class="incremental">
<li><p>What if you were running your python program on a cretin system that didn't give you enough memory?</p></li>
<li><p>But you knew the kernel had access to plenty more...</p></li>
<li><p>You <strong>could</strong> fire off a socket and use it as some FIFO form of memory storage...</p></li>
<div class="section slide level1" id="extremely-contrived-example">
<h1 id="extremely-contrived-example">Extremely Contrived Example</h1>
<div class="figure">
<img src="images/memory_example.png" /><p class="caption"></p>
<ul class="incremental">
<li>Main thread wants to &quot;save some data&quot; out of process memory by creating a local socket with a large buffer</li>
<li>the consumer thread just creates the socket/s</li>
<li>to illustrate the concept my worker thread isn't allowed to access this memory (read from socket) until all the data has been stored</li>
<li>sha256 on the bytes saved and recalled should match</li>
<div class="section slide level1" id="example-socket_memory.py-output">
<h1 id="example-socket_memory.py-output">example socket_memory.py output</h1>
<pre><code>Connected by (&#39;;, 45527)
waiting for producer to finish filling the socket
&quot;Saved&quot; 250 blocks of 2048 random bytes
SHA256 of sent: 290491d7b26ab55cf62cf9ea662634b0bd63509dfbaf02877aab458a161a5fd9
okay lets have your data
Received 250 x 2048 bytes of data
SHA256 of recv: 290491d7b26ab55cf62cf9ea662634b0bd63509dfbaf02877aab458a161a5fd9</code></pre>
<ul class="incremental">
<li>No I'm not really serious, but it did appear to work...</li>
<div class="section slide level1" id="new-toy">
<h1 id="new-toy">New toy</h1>
<div class="figure">
<img src="images/raspi.png" /><p class="caption"></p>
<ul class="incremental">
<li>The much awaited Rasberry Pi! - actually I'm <strong>still</strong> waiting for mine</li>
<li>easily fitted with a USB bluetooth dongle</li>
<li>actually just another computer...</li>
<li>which is a bit boring</li>
<li>need something more fun</li>
<div class="section slide level1" id="newer-toy">
<h1 id="newer-toy">New(er) toy!</h1>
<div class="figure">
<img src="images/iRacer.jpg" /><p class="caption"></p>
<p>The <strong>iRacer</strong> from sparkfun.</p>
<div class="section slide level1" id="iracer-features">
<h1 id="iracer-features">iRacer features</h1>
<ul class="incremental">
<li>on-board Bluetooth radio</li>
<li>RFCOMM Bluetooth...</li>
<li>Protocol is dead easy - <strong>0xXY</strong> where X is the direction and Y is the speed. E.G 0x16: Direction=Forwards, mid-speed.</li>
<div class="section slide level1" id="bluetooth">
<h1 id="bluetooth">Bluetooth</h1>
<ul class="incremental">
<li><strong>One</strong> mention of &quot;bluetooth&quot; on the Python 3.3 socket docs... <img src="images/bluetooth_documentation.png" /></li>
<li>I mean how hard could it be...?</li>
<li>To be fair I had to use the source</li>
<div class="section slide level1" id="bluetooth-controlled-car">
<h1 id="bluetooth-controlled-car">Bluetooth controlled Car</h1>
<pre class="sourceCode python"><code class="sourceCode python">    <span class="ch">import</span> socket
    <span class="ch">import</span> time

    <span class="kw">class</span> BluetoothCar:
        <span class="kw">def</span> <span class="ot">__init__</span>(<span class="ot">self</span>, mac_address=<span class="st">&quot;00:12:05:XX:XX:XX&quot;</span>):
            <span class="ot">self</span>.socket = socket.socket(

            <span class="ot">self</span>.socket.<span class="ot">connect</span>((mac_address, <span class="dv">1</span>))

        <span class="kw">def</span> forwards(<span class="ot">self</span>, duration=<span class="fl">1.0</span>):
            <span class="ot">self</span>.drive(<span class="bn">0x16</span>, duration)

        <span class="kw">def</span> drive(<span class="ot">self</span>, command, duration):
            <span class="ot">self</span>.socket.send(bytes([command]))
            <span class="ot">self</span>.socket.send(bytes([<span class="bn">0x00</span>]))</code></pre>
<div class="section slide level1" id="can">
<h1 id="can">CAN</h1>
<ul class="incremental">
<li>The <strong>C</strong>ontroller <strong>A</strong>rea <strong>N</strong>etwork is a bus standard designed to allow microcontrollers and devices to communicate with each other.</li>
<li>It has priority based bus arbitration, reliable deterministic communication.</li>
<li>Used in cars, trucks, wheelchairs... learn more at <a href="http://en.wikipedia.org/wiki/CAN_bus"><code class="url">http://en.wikipedia.org/wiki/CAN_bus</code></a>.</li>
<p>SocketCAN is a set of open source CAN drivers and a networking stack contributed by Volkswagen Research to the Linux kernel.</p>
<div class="section slide level1" id="virtual-can">
<h1 id="virtual-can">Virtual Can</h1>
<p>On reasonably modern GNU/Linux systems open a virtual can interface:</p>
<pre><code>sudo modprobe vcan
sudo ip link add dev vcan0 type vcan
sudo ifconfig vcan0 up</code></pre>
<h2 id="socket-can">Socket Can</h2>
<p>With the hardware and appropriate socketcan kernel module, open a real CAN device:</p>
<pre><code>sudo ip link set can0 up type can bitrate 1000000</code></pre>
<ul class="incremental">
<li>Allows very easy filtering in the kernel (fast)</li>
<div class="section slide level1" id="python-and-socketcan">
<h1 id="python-and-socketcan">Python and SocketCan</h1>
<pre><code># create a raw socket
s = socket.socket(socket.AF_CAN, socket.SOCK_RAW, socket.CAN_RAW)

# We can only filter on just the bits of the can_id in this mask:
can_mask = 0x00F0
# Create the filter
rfilter = struct.pack(can_filter_fmt, can_id, can_mask)

# Pass those filter options to the kernel via socket.setsockopt
s.setsockopt(socket.SOL_CAN_RAW, socket.CAN_RAW_FILTER, rfilter)

#bind our CAN socket to the &#39;can0&#39; network interface
<div class="section slide level1" id="elua">
<h1 id="elua">eLua</h1>
<div class="figure">
<img src="images/logoeLua.png" /><p class="caption"></p>
<ul class="incremental">
<li>full implementation of the Lua Programming Language for the embedded world</li>
<li>eLua runs on the &quot;bare-metal&quot;</li>
<li>awesome for prototype and experiments on a Rapid Application Develop model</li>
<div class="section slide level1" id="lm3s-eval-board">
<h1 id="lm3s-eval-board">LM3S Eval Board</h1>
<div class="figure">
<img src="images/elua.jpg" /><p class="caption"></p>
<ul class="incremental">
<li>Has CAN</li>
<li>And a simple serial interface</li>
<p>To start my eLua dev board spamming CAN messages: miniterm.py --port=/dev/ttyUSB0 --baud=115200</p>
<pre><code>lua /mmc/bcan.lua</code></pre>
<p>Then on the PC run wireshark or my real_raw_can.py demo.</p>
<div class="section slide level1" id="sockets-for-broadcast-manager-bcm">
<h1 id="sockets-for-broadcast-manager-bcm">Sockets for Broadcast-Manager (BCM)</h1>
<ul class="incremental">
<li>The Broadcast-Manager provides functions to send messages on the CAN bus once or periodically, as well as notify of changes in received CAN frames, recognizing specific CAN IDs.</li>
<li>I.E. Black Magic</li>
<li>Least documented thing I've ever seen - http://www.brownhat.org/docs/socketcan/llcf-api.html <img src="images/bcm_patch.png" /></li>