esp8266 webImport

UPDATED and BUGFIXED!

MicroPython Repl command line is very nice, because you can telnet to the chip and read-and-evaluate python code. It is a great way of learning embedded IoT. The bad thing you cannot copy file while it is running, so experimenting get bad.

Also an automatic module update would be a very nice thing to have.

So I have created a small module called webImport which is able to

1. download a module.py from a known http server
2. write it in the esp8266 and import it
3. enable your esp8266 to be hacked by a worm :) (see below for caveats)

The idea is to serve your remote module via a HTTP server and load it on the repl (or programmatically) with

from webimport import web_import
web_import('modulename')

After a while you edit your modulename.py and you make a newMethod. So you want to reload it...with...

web_import('modulename')
modulename.method()

The CODE

The code is below and was tested on esp8266. It should work on Pyboard and more powerful board too.

# Implement a web import api
# Rev 1.0.2
# Rev 1.0.1  was bugged 
#import ujson as json
import uos as os
import usocket as socket
debug=True
def say(m):
if debug:
print("wi: "+str(m))
def extractHeaders(headers_pair2split): 

hp=headers_pair2split.split(b'\r\n')
headers={}
for elem in hp:
k,v=elem.split(b':',1)
headers[k]=v
say(str(k)+" => "+str(v))
def http_get_async(url):
say("webImport:"+url)
_, _, host, path = url.split('/', 3)
if ':' in host:
host, port = host.split(':')
else:
port = 80
addr = socket.getaddrinfo(host, int(port))[0][-1]
s = socket.socket()
s.connect(addr)
s.send(bytes('GET /%s HTTP/1.0\r\nHost: %s\r\n\r\n' % (path, host),
'utf8'))
headerStr = b''
body_bytes= b''
buffer=b''
## Skip headerStr: they are tiny, so we get a small chunk
while True:     

buffer= buffer + s.recv(250)
if b'\r\n\r\n' in buffer:
break
# Evict headerStr
headerStr, body_bytes = buffer.split(b'\r\n\r\n',1)
# GG Consider splitting via b'\r\n' to get headers, get Date:
# to implement a caching algorithm
#print("wi: debug: headers:"+str(headerStr)+" SIZE:"+str(len(str(headerStr))))
response, headers_pair2split=headerStr.split(b'\r\n',1)
extractHeaders(headers_pair2split)
if '200 OK' not in response:
raise ImportError("Response not ok:"+str(response))
yield body_bytes
while True:
data= s.recv(1600)
if data:
yield data
else:
break
def http_get(url):
response = b''
try:
get = http_get_async(url)
while True:
file_bytes = get.send(None)
response += file_bytes
except StopIteration:
pass
response_str = str(response, 'utf-8')
return response_str

def ensure_dirs(path):
split_path = path.split('/')
if len(split_path) > 1:
for i, fragment in enumerate(split_path):
parent = '/'.join(split_path[:-i])
try:
os.mkdir(parent)
except OSError:
pass
def http_get_to_file(url, path):
ensure_dirs(path)
totSize=0
with open(path, 'w') as outfile:
try:
get = http_get_async(url)
while True:
file_bytes = get.send(None)
outfile.write(file_bytes)
totSize+=len(file_bytes)
except StopIteration:         

outfile.close()
say("Loaded "+path+"#"+str(totSize))
def web_import(moduleName, host="192.168.1.7:8000", on_error_use_cache=True):
try:
fname="./"+moduleName+".py"
url="http://"+host+"/"+fname
http_get_to_file(url,fname)
return import(moduleName)
except OSError as e:
print(str(e))
if on_error_use_cache:                     

m= import(moduleName)
print("wi: "+moduleName+" ECONN cache ok")
return m

 

Optimization

As a bonus you can cross-compile the module, using the MicroPython cross compiler, which runs under any Unix-like system and compiles .py scripts into .mpy files.

https://github.com/micropython/micropython/tree/master/mpy-cross

This step is optional but offer to you less memory usage and better performance

 

Final thoughts

Be warned: having a remote download and execution code is a security risk. Node.js' Npm package manager face this risk again and again, so you should consider some sort of password protection and / or https authentication using the ussl module which does not protect you but Man in the middle attack.