Skip to content

GitLab

B
B.I.T.S. Server
Source
Select Git revision
...
Target
Select Git revision
 
Commits (2)
Hide whitespace changes
Inline Side-by-side
Showing with 18 additions and 682 deletions
INFO.md deleted 100644 → 0
View file @ 6b7cd5bd
Cos'è bits
==========
BITS è un indicatore in tempo reale dello stato della sede del
[POuL](http://www.poul.org). È stato progettato per sapere se c'è qualcuno
in sede senza dover andare a controllare di persona, consultando direttamente
il proprio portatile o smartphone.
L'architettura di BITS è composta da un Arduino, una Fonera, e un web server.
![](/static/bits1_small.jpg)
![](/static/bits2_small.jpg)
Il codice sorgente è scritto in Python e C++ ed è Free Software/Open Source,
disponibile su [GitHub](https://github.com/esseks/bitsd).
Stato della sede
================
Questa è la funzione principale del sito BITS: informare in tempo reale se la
sede è aperta o chiusa. Viene anche indicata la data e l'ora dell'ultima volta
in cui la sede è stata aperta/chiusa.
Non è necessario ricaricare la pagina per avere un informazione aggiornata,
tramite l'utilizzo di WebSocket la pagina si aggiorna da sola.
Previsione presenze
-------------------
La sede del POuL non ha orari prefissati, è aperta quando almeno uno dei suoi
membri decide di farci un salto. Di solito questo avviene in corrispondenza
delle ore buche tra una lezione e l'altra.
Poichè le ora buche sono uguali tutte le settimane all'interno di un semestre,
al sito è stata aggiunta una previsione della presenza di persone in sede,
basata sullo storico delle presenze nelle settimane passate.
Le previsioni sono rappresentate graficamente in una tabella che, per ogni ora
e giorno della settimana, mostra la probabilità che la sede sia aperta tramite
un colore (rosso=bassa probabilità, verde=alta probabilità).
![Grafico delle presenze](/bits_presence.png)
Temperatura in sede
-------------------
BITS è anche dotato di un sensore che registra la temperatura in sede.
Storico presenze
----------------
Le informazioni sulle presenze in sede sono salvate in un database, usato per
la previsione delle presenze ma anche [direttamente accessibile](/data).
\ No newline at end of file
MANIFEST.in deleted 100644 → 0
View file @ 6b7cd5bd
include AUTHORS
include README.md
include COPYING
\ No newline at end of file
README.md
View file @ 2ccf455f
......@@ -9,149 +9,25 @@ Processed information is made available both in human readable HTML
and machine friendly JSON and it is consumed via a large variety of clients:
web browsers, native mobile apps, browser plugins, desktop widgets...
Note
----
Most of the URLs in this page refer to a running instance of BITS server, so they
will not work when viewing this document on GitHub (and should not, since
BITS server is not installed yet).
Architecture overview
=====================
BITS server is completed by two hardware components, forming the 3-tier
BITS architecture:
1. A STm32 ARM Cortex board connected to an LCD display for messages/
temperature/presence updates, a button for signaling open/closed statuses
and various environmental sensors.
2. A Fonera bridging STm32 to the server via I2C (cortex-fonera)
and a VPN (fonera-server). The VPN setup was necessary to let BITS work
on our campus wifi link, which is heavily filtered.
3. A web server (this!) receiving update commands via BITS-protocol,
as detailed in the docs, and displaying public data to web clients in pull
or push mode (true realtime).
The Cortex board is powered by Miosix OS and a daemon, both written
by that crazy genius of beta4 together with the C++ BITS developer team.
Find the code on [GitHub](https://github.com/Otacon22/bits).
This web server
===============
BITS server is written in Python and is built on Tornado. It was created with a few
objectives in mind:
1. Handling requests via events (epoll on Linux) in a single process, instead of
forking instances, to leverage server load and work efficiently on single
core machines -- like our current server host.
2. Handling the mighty c10k storm (and, in general, several long polling connections).
3. Serving dynamic data with minimum overhead and bandwidth consumption.
4. Enabling real time presence notifications via Web Socket push (and gracefully
falling back to AJAX pull on older browsers).
5. Being as modular as possible.
Point _5_ is the actual reason why this project was started, as the legacy
architecture had grown around a (very smart, to say all) kludge and was deemed
to be unmaintainable.
Actually, BITS server is composed by three components:
Web Server
----------
Serving five page models:
1. [Homepage](/), which is just a skeleton: data retrieval is handled client side.
2. A [paginated history](/log), displaying information logged in the DB.
3. A mini wiki engine based on markdown. Actually, this README is rendered as the
[info page](/info) and more informative pages are to come.
4. [Recent data](/data), in a machine parseable form (JSON).t
5. [Status](/status) as a single digit (0=closed, 1=open), for building a minimal
desktop/mobile widget.
Information on the homepage includes **presence status** (e.g. when it was
opened or closed), **current temperature** and plots of historical values.
Web Socket server
-----------------
Keeping a list of connected clients and broadcasting updates as soon as
they are processed.
BITS-miniprotocol server
------------------------
A raw TCP text-based protocol (detailed in the docs) allows
to send and receive data or commands to and from the Fonera.
This server is bound to a private address, so that only the VPN internal hosts
will be able to log data and change status.
TODOs
=====
BITS server is completed by multiple hardware components:
MAC address detection
---------------------
Upon detection of a registered POuL member's MAC address, for more than a certain
amount of time, the Fonera will deduce that someone has arrived and flag the status
as open. Same for closing.
Notes
=====
External code
-------------
Assets in `BITS server/server/http/assets` have been imported from legacy project.
Some JS scripts are libraries licensed under the terms specified in the respective
files, the other JS scripts and the CSS files had been coded by
[thypon](https://github.com/thypon) for the legacy BITS project.
Appeareance
-----------
The style of html pages has been imported from the previous versions.
A restyle is pending and should be straightforward, as everything (including
those pre-2000 `|` breadcrumb bars) is rendered with some CSS3 magic.
We strove to keep markup as clean and semantic as possible.
1. A esp32-based mcu ([code](https://gitlab.poul.org/b4/bits-button)) running micropython, which listens for button presses and sends MQTT messages.
2. A computer running inside headquartes running a MQTT broker ([mosquitto](https://mosquitto.org/)) which the esp32 connects to.
3. A web server (this!) receiving status updates via MQTT,
and displaying public data to web clients in pull or push mode.
Bootstrap your instance
=======================
TL;DR
-----
First create a Python environment with all libraries inside with:
$ make virtualenv
Activate it with:
$ . env/bin/activate
If this is the first time the daemon is run or if the DB has been reset, issue:
$ ./bootstrap.py
Then start the daemon with:
$ ./bitsd.py
When developing, you will find particurarly useful `--developer_mode` and
`--log_queries` command line options (see more below).
Requirements
------------
BITS is developed on Python 2.7. Python3k is not supported nor we look
forward to, although we will switch to it at some point in the future.
BITS is developed on Python 3.
Hard dependencies are:
......@@ -195,6 +71,11 @@ Users can be added, removed and modified using `./usermanage.py` script:
$ ./usermanage.py modify test
New password for `test`:
Docker
------
BITS comes with a Dockerfile that can be used to build a bits image for production purposes
Development
===========
......@@ -217,31 +98,10 @@ trust me.
GIT workflow
------------
The upstream repository has two main branches:
* **master** is the stable branch.
1. _no branch_ shall be merged directly into `master`
2. _only commits_ that can be pushed directly on this branch are typos
and extremely urgent bugfixes. Anything else shall be merged in `development`
(see below).
3. since all changes committed to `master` are failsafe, changes can be
merged into another branch anytime.
* **development** is the branche where (guess what?) development is carried on.
**this code is not guaranteed to be safe for production** or even to execute.
1. _new features_ will be developed in branches of `development`, then merged
back when ready.
2. _merge_ to `development` can happen as soon as the new feature is considered
ready.
3. when code in `development` has been deemed stable, it can be merged into
`master`.
Never push directly into upstream repository. Instead, fork the repo on GitHub,
develop according to the workflow above in your fork and then file a pull
request as soon as you have a changeset ready.
This way, it will be easy to track blocks of commits and features introduced.
Bugs and patches
----------------
This project is hosted on [GitHub](https://github.com/esseks/BITS server), you
are welcome to use the bug tracker, wishlist and make pull requests.
**master** is the stable branch
1. Feature branches are merged to master once they are tested
2. Nothing is committed directly to master, except for:
* Documentation updates
* Urgent Bugfixes
3. Branches/PR are welcome! Ask on mailinglist@poul.org for access if you don't have it already
bitsd.conf
View file @ 2ccf455f
......@@ -18,18 +18,6 @@ This is also a valid Python file, but don't abuse that.
## Unix socket the web server will be listening on.
#web_usocket = ''
## Port for fonera server. Only if ws_usocket is not defined.
#control_local_port = 8888
## Unix socket the remote control will be listening on.
#control_local_usocket = ''
## The address the remote control will be bound to.
#control_local_address = "127.0.0.1"
## The address of the remote control unit (Fonera).
#control_remote_address = "127.0.0.1"
## UID to chown the unix sockets to
#usocket_uid = 1000
......
bitsd/client/__init__.py deleted 100644 → 0
View file @ 6b7cd5bd
#
# Copyright (C) 2013 Stefano Sanfilippo
# Copyright (C) 2013 BITS development team
#
# This file is part of bitsd, which is released under the terms of
# GNU GPLv3. See COPYING at top level for more information.
#
"""
Client proxies. Submodules will handle
"""
\ No newline at end of file
bitsd/listener/__init__.py deleted 100644 → 0
View file @ 6b7cd5bd
#
# Copyright (C) 2013 Stefano Sanfilippo
# Copyright (C) 2013 BITS development team
#
# This file is part of bitsd, which is released under the terms of
# GNU GPLv3. See COPYING at top level for more information.
#
"""
Listeners are server components waiting for commands on given
ports/hosts/address or events (on the countrary, a server will actually
*serve* content to the client).
"""
from tornado.options import options
from .handlers import RemoteListener
from bitsd.common import bind, LOG
from . import hooks
def start():
"""Connect and bind listeners. **MUST** be called at startup."""
__inject_broadcast()
fonera = RemoteListener()
LOG.info('Starting remote control...')
LOG.info(
'My IP address is {}, remote IP address is {}'.format(
options.control_local_address,
options.control_remote_address
)
)
bind(
fonera,
options.control_local_port,
options.control_local_usocket,
address=options.control_local_address
)
def __inject_broadcast():
"""Lazily load broadcast() function to break circular dependencies"""
from bitsd.server.handlers import broadcast
hooks.broadcast = broadcast
bitsd/listener/handlers.py deleted 100644 → 0
View file @ 6b7cd5bd
#
# Copyright (C) 2013 Stefano Sanfilippo
# Copyright (C) 2013 BITS development team
#
# This file is part of bitsd, which is released under the terms of
# GNU GPLv3. See COPYING at top level for more information.
#
"""
TCP server receiving raw messages and invoke correct handlers
(from module `.hooks`). Listens for remote commands on BITS-miniprotocol
and dispatches to Fonera via bitsd.client.fonera.Fonera proxy.
"""
import tornado.tcpserver
from tornado.options import options
from tornado.iostream import StreamClosedError
from bitsd.common import LOG
from .hooks import *
def send(string):
if RemoteListener.STREAM is None:
LOG.error("No Fonera connected! Not sending %r", string)
return
try:
RemoteListener.STREAM.write(string.encode('utf-8'))
except StreamClosedError as error:
LOG.error('Could not push message to Fonera! %s', error)
class RemoteListener(tornado.tcpserver.TCPServer):
"""
Handle incoming commands via BITS mini protocol.
Trying to do something KISS.
Commands are rpc-like: function name and eventual args, separated by spaces.
One command per line, "\\n" as line separator.
Numeric argument are printed as-is, string arguments are encoded in base64.
**status** <int>
Parameter 0 is "closed", 1 is "open".
>>> "status 1\\n"
**enter** <int>
One person is added to the list of persone in sede.
The first parameter is the number inserted on the numeric keypad.
>>> "enter 5\\n"
**leave** <int>
One person is removed from the list of persone in sede.
The first parameter is the number inserted on the numeric keypad.
>>> "leave 5\\n"
**message** <string>
A message is added to the list of messages shown on the display.
>>> "message bG9sCg==\\n"
**sound** <int>
Play a sound on the fonera.
The parameter is an index into a list of predefined
sounds. Sad trombone anyone?
>>> "sound 0\\n"
"""
ACTIONS = {
b'temperature': handle_temperature_command,
b'status': handle_status_command,
b'enter': handle_enter_command,
b'leave': handle_leave_command,
b'message': handle_message_command,
b'sound': handle_sound_command,
}
STREAM = None
def __init__(self):
super(RemoteListener, self).__init__()
def handle_stream(self, stream, address):
"""Handles inbound TCP connections asynchronously."""
LOG.info("New connection from Fonera.")
if address[0] != options.control_remote_address:
LOG.error(
"Connection from `%s`, expected from `%s`. Ignoring.",
address,
options.control_remote_address
)
return
if RemoteListener.STREAM is not None:
LOG.warning("Another connection was open, closing the previous one.")
RemoteListener.STREAM.close()
RemoteListener.STREAM = stream
RemoteListener.STREAM.read_until(b'\n', self.handle_command)
def handle_command(self, command):
"""Reacts to received commands (callback).
Will separate args and call appropriate handlers."""
# Meanwhile, go on with commands...
RemoteListener.STREAM.read_until(b'\n', self.handle_command)
command = command.strip(b'\n')
if command:
args = command.split(b' ')
action = args[0]
try:
handler = RemoteListener.ACTIONS[action]
except KeyError:
LOG.warning('Remote received unknown command `%s`', args)
else:
# Execute handler (index 0) with args (index 1->end)
try:
handler(*args[1:])
except TypeError:
LOG.error(
'Command `%s` called with wrong number of args', action
)
else:
LOG.warning('Remote received empty command.')
bitsd/listener/hooks.py deleted 100644 → 0
View file @ 6b7cd5bd
#
# Copyright (C) 2013 Stefano Sanfilippo
# Copyright (C) 2013 BITS development team
#
# This file is part of bitsd, which is released under the terms of
# GNU GPLv3. See COPYING at top level for more information.
#
"""
Hooks called by `.handlers` to handle specific commands.
"""
# NOTE: don't forget to register your handler in RemoteListener.ACTIONS
# : and in __all__ below!!
import base64
from bitsd.listener import notifier
from bitsd.persistence.engine import session_scope
from bitsd.persistence.models import Status
import bitsd.persistence.query as query
from bitsd.common import LOG
#: This will be initialized by bitsd.listener.start()
broadcast = None
__all__ = [
'handle_temperature_command',
'handle_status_command',
'handle_enter_command',
'handle_leave_command',
'handle_message_command',
'handle_sound_command'
]
def handle_temperature_command(sensorid, value):
"""Receives and log data received from remote sensor."""
LOG.info('Received temperature: sensorid=%r, value=%r', sensorid, value)
try:
sensorid = int(sensorid)
value = float(value)
except ValueError:
LOG.error('Wrong type for parameters in temperature command!')
return
with session_scope() as session:
temp = query.log_temperature(session, value, sensorid, 'BITS')
broadcast(temp.jsondict())
def handle_status_command(status):
"""Update status.
Will reject two identical and consecutive updates
(prevents opening when already open and vice-versa)."""
LOG.info('Received status: %r', status)
try:
status = int(status)
except ValueError:
LOG.error('Wrong type for parameters in temperature command')
return
if status not in (0, 1):
LOG.error('Non existent status %r, ignoring.', status)
return
textstatus = Status.OPEN if status == 1 else Status.CLOSED
with session_scope() as session:
curstatus = query.get_current_status(session)
if curstatus is None or curstatus.value != textstatus:
status = query.log_status(session, textstatus, 'BITS')
broadcast(status.jsondict())
notifier.send_status(textstatus)
else:
LOG.error('BITS already open/closed! Ignoring.')
def handle_enter_command(userid):
"""Handles signal triggered when a new user enters."""
LOG.info('Received enter command: id=%r', userid)
try:
userid = int(userid)
except ValueError:
LOG.error('Wrong type for parameters in temperature command!')
return
LOG.error('handle_enter_command not implemented.')
def handle_leave_command(userid):
"""Handles signal triggered when a known user leaves."""
LOG.info('Received leave command: id=%r', userid)
try:
userid = int(userid)
except ValueError:
LOG.error('Wrong type for parameters in temperature command!')
return
LOG.error('handle_leave_command not implemented.')
def handle_message_command(message):
"""Handles message broadcast requests."""
LOG.info('Received message command: message=%r', message)
try:
decodedmex = base64.b64decode(message)
except TypeError:
LOG.error('Received message is not valid base64: %r', message)
else:
text = decodedmex.decode('utf8')
#FIXME maybe get author ID from message?
user = "BITS"
with session_scope() as session:
user = query.get_user(session, user)
if not user:
LOG.error("Non-existent user %r, not logging message.", user)
return
message = query.log_message(session, user, text)
broadcast(message.jsondict())
notifier.send_message(text)
def handle_sound_command(soundid):
"""Handles requests to play a sound."""
LOG.info('Received sound command: id=%r', soundid)
try:
soundid = int(soundid)
except ValueError:
LOG.error('Wrong type for parameters in temperature command!')
return
else:
notifier.send_sound(soundid)
bitsd/listener/notifier.py deleted 100644 → 0
View file @ 6b7cd5bd
#
# Copyright (C) 2013 Stefano Sanfilippo
# Copyright (C) 2013 BITS development team
#
# This file is part of bitsd, which is released under the terms of
# GNU GPLv3. See COPYING at top level for more information.
#
"""
Send BITS-miniprotocol notification messages to remote host.
"""
import base64
from bitsd.persistence.models import Status
from . import handlers
def send_message(text):
"""
A message is added to the list of messages shown on the Fonera display.
"""
handlers.send("message {}\n".format(base64.b64encode(text.encode('utf-8'))))
def send_status(value):
"""
Send open or close status to the BITS Fonera.
Status can be either 0 / 1 or Status.CLOSED / Status.OPEN
"""
try:
value = int(value)
except ValueError:
value = 1 if value == Status.OPEN else 0
handlers.send("status {}\n".format(value))
def send_sound(soundid):
"""
Play a sound on the fonera.
The parameter is an index into a list of predefined sounds.
Sad trombone anyone?
"""
handlers.send("sound {}\n".format(soundid))
bitsd/main.py
View file @ 2ccf455f
......@@ -14,7 +14,6 @@ BITSd entry point and unix signal handlers.
import bitsd.properties
import bitsd.server as server
import bitsd.listener as listener
import bitsd.mqtt_listener as mqtt_listener
import bitsd.persistence as persistence
......@@ -67,7 +66,6 @@ def main():
persistence.start()
server.start()
#listener.start()
mqtt_listener.start()
# Add signal handlers...
......
bitsd/properties.py
View file @ 2ccf455f
......@@ -23,29 +23,6 @@ define("web_usocket",
group="Networking"
)
define("control_local_port",
default=8888,
help="Port for fonera server. Only if ws_usocket is not defined.",
group='Networking'
)
define("control_local_usocket",
default='', help="Unix socket the remote control will be listening on.",
group="Networking"
)
define("control_local_address",
default="127.0.0.1",
help="The address the remote control will be bound to.",
group='Networking'
)
define("control_remote_address",
default="127.0.0.1",
help="The address of the remote control unit (Fonera).",
group="Networking"
)
define("reverse_proxied",
default=False,
help="Get remote IP address via X- headers (use when reverse proxied).",
......
bitsd/test/__init__.py deleted 100644 → 0
View file @ 6b7cd5bd
#
# Copyright (C) 2013 Stefano Sanfilippo
# Copyright (C) 2013 BITS development team
#
# This file is part of bitsd, which is released under the terms of
# GNU GPLv3. See COPYING at top level for more information.
#
"""
Unit tests package
"""
\ No newline at end of file
requirements.txt
View file @ 2ccf455f
......@@ -3,7 +3,6 @@ markdown
sqlalchemy>=0.7
tornado==4.*
paho-mqtt
#mysqlclient
psycopg2
svgwrite~=1.3
colour~=0.1
......
setup.cfg deleted 100644 → 0
View file @ 6b7cd5bd
[bdist_rpm]
release = 1
packager = Stefano Sanfilippo <a.little.coder@gmail.com>
doc_files = README.md AUTHORS COPYING
requires = python-tornado python-sqlalchemy python-passlib python-concurrent.futures
setup.py deleted 100755 → 0
View file @ 6b7cd5bd
#! /usr/bin/env python3
#
# Copyright (C) 2013 Stefano Sanfilippo
# Copyright (C) 2013 BITS development team
#
# This file is part of bitsd, which is released under the terms of
# GNU GPLv3. See COPYING at top level for more information.
#
from setuptools import setup
setup(
name='bitsd',
version='2.1',
license='GPLv3',
description='Presence server, logger and remote listener.',
author='Stefano Sanfilippo et al.',
author_email='a.little.coder@gmail.com',
url='https://github.com/esseks/bitsd',
install_requires=[
'tornado >= 2.3',
'sqlalchemy >= 0.7',
'markdown',
'futures',
'pycares'
],
packages=[
'bitsd',
'bitsd.client',
'bitsd.listener',
'bitsd.persistence',
'bitsd.server',
'bitsd.test',
],
scripts=[
'bitsd.py',
'usermanage.py',
'bootstrap.py'
],
package_data={
'bitsd.server': [
'templates/*.html',
'assets/*',
'assets/lib/*',
'scripts/*'
],
},
data_files=[
('etc', [
'bitsd.conf'
])
]
)