Shakaran

Several months have passed since I wrote about Freestation. I would like show some advances and write about my dissertation project that I hope to finish for the 2th June as last day for deadline on this course.

An introduction reminder

FreeStation is a program built for help to distribute easy and reliable software and information on point of interest (POI).

Basically, a POI is huge device that you can find on universities, libraries, hospitals, police stations, train stations, etc.

For example, this are a couple of images of POIs:

DNI POI on police station

This is POI on a police station for perform changes with a DNI (National Idenfication Document on Spain). It shows a keyboard, trackball and DNI card reader for easy operations with credential authentication using DNI.

It has normal power connection but it also has internet connection for load data. On the following image:

Front detail of DNI POI on a police station

As you can apreciate, it shows with more details a screen showing input boxes for the user and a simple interface with buttons. The height POI is adjusted to people in wheelchairs (a laudable feature for accessibility).

Another example of POI:

Adif POI on a train station

This image corresponds to another POI on a Adif’s train station. It has a different design, totally vertical with a big screen rotated. Unfortunaly when I take the photo it was turned off, but I think that it is a tactil screen model with internet connection too.

It incorporates a audio system maybe useful for blind people and as possible confirmation for using audio notification. Sadly, it is enough high for wheelchairs people, so maybe the only functions could be a information panel with low interaction. I can’t ensure this, because I never can see running this POI when I travel.

Finally another radical example of POI (three different examples should be enough for ilustrate my point):

SESCAM POI on a hospital

This is another interesting POI that has a intercommunication phone for direct assistance, keyboard, tracball mouse and sanitary card reader. It also has a tray as printer for deliver useful data to the user. Sadly, when I take the photo the POI also is turned off.

So, this is only a reduced set of examples of a huge variety of POIs. Usually you can find the same model for each same hospital, train station, police station, university, etc. Each model runs practically the same model of software and hardware features. But, when the time past, it is highly probably that some POIs have breakdows, energy loss, software bugs, unconsistence of versions, etc.

So, FreeStation try to repair and unify this problems for offer a kindly solution, because it is important deploy same versions between POIs, recieve notifications when a POI it is offline for a breakdown, auto-restart the POI if some hard bug happens, etc.

Architecture

The Freestation architecture is mainly based on client-server model. Each component is easy to tweak or customize and it doesn’t require technical knowledge for diary use.

Architecture - Server and clients infrastructure

Let me introduce you the architecture with real screenshots of the application and some diagrams.

Server architecture

FreeStation server runs over a software stack consisting of several technologies and components. It handles all the communication with every client.

It has a friendly webserver GUI panel administration based on PHP + Javascript.

FreeStation Server Webserver GUI

As you can see, you can start/stop/restart the server, see real status and different useful output logs for analyce and diagnose whatever event.

It allows easily make the common tasks for deploy and configure a high amount of clients with POIs.

Under the webserver GUI, it runs a daemon process as backend based on python. Each action and task is handled internally and dispatched on fast way.

The backend can make asynchronous communication with clients. For that the backend uses ICE (Internet Comunication Engine of ZeroIce).

This allows a reliable and robust way to transfer huge information datasets between a big amount of clients and the server.

For example, you also can see a list of current clients handled by the server with stats:

FreeStation Server GUI - List of clients

The list shows each client hostname, IP address, last connection, requests, status and some actions for each client. You can add/remove/edit clients and all the data associated with a clients would be updated.

Client architecture

A client run only on a POI and periodically poll to the server for updates or information. Clients need a specification for deploy widgets and the data associated for each widget.

A widget on Freestation terminology is a abstract element that performs a atomic operation and provides communication for the data handled inside the widget.

For complex operations a widget could associate with more widgets for perform a sofisticated task or simply commnicate data with other widgets. The widgets only run on clients, but all them are configured only from a server.

This describe a orchestra model where the server acts like a manager and clients with widgets acts like a choir.

That means that a client only can communicate with the server and it is not allowed the communication between clients. But this can be easily a SPOF (Single Point Of Failure) for server. So, it is planned on a future allow several masters servers as failover and syncronize and replicate data between masters.

In other hand, an a server, you have a list of widgets available to configure on a client.

FreeStation Server GUI - List of total widgets

For example, you can choose deploy and configure a USB Storage widget and a Mount Device widget on a client. That widgets could associate operations for list several files of books about medicine. When the user pick a set and choose save to a usb key, the Mount Device detect the usb key and usb storage widget write the data. The result it is a happy user that obtain the needed information quickly and without problems.

FreeStation Server - Associate widget to client

You can associate the needed widgets for a client and configure/edit each widget with custom params (for example, only allow 5 items for save on Mount Device):

FreeStation Server - Configure Widget for a client

Another example, you can choose a Video widget and display a video on POI for idle mode. When the user touch or interact with the POI, you can load a Browser Widget with a concrete webpage. Additionally as some institution you can choose a simple logo widget for display your institution logo on screen.

When a client is totally configured, then can be deployed on POI (normal desktop computer customized). A simple example of client screen deployed:

FreeStation Client - Welcome wigdets configured

There are not limit defining or using new widgets. So in theory, the widget model scale for each custom need for a institution.

Free & easy to scale to your needs

With FreeStation some basic widgets are offered, but on the future, it could has repository widgets associated, widget patners and offer a rich market for developers that they are interested on develop custom widgets for companies, institutions, hospitals.

Soon, I plan to release the code as AGPLv3 on Github. Still I need polish some widgets, finish a real demo case for my dissertation and literally write a boring book with approximattelly 200 pages (currently, I have around 80 bad structure pages written).

So, the project is nearly to finish and with lucky I can finish before to 2th june deadline (and 15 th june official presentation) and finally has a big engineer diploma on my wall, yay!

Meanwhile, there are nearly two months of hard work, But shhh … I found a secret weapon

Secret hard work weapon

Por lo general, al iniciar alguna consola o terminal bash, se leen muchos archivos de configuración por defecto y se abren archivos para guardar los eventos de actividad que van ocurriendo. Aunque es útil, puede ralentizar un poco la carga del terminal.

Normalmente, cuando un sistema lleva mucho tiempo en ejecución o ha pasado bastante tiempo desde que fue instalado por primera vez, se acumulan bastantes logs de actividad de diferentes procesos.

Logrotate es una buena utilidad, activada por defecto en la mayoría de distribuciones para los archivos más importantes y que generan mayor volumen de información. Es muy útil porque ademas comprime los logs y los borra pasado un tiempo.

El problema, es que no todas las configuraciones por defecto de logrotate suministradas por la distribución, pueden resultar útiles para todos los usuarios.

Aunque se puede optar por personalizar logrotate para los archivos necesarios, muchas veces se desconocen cuales son todos los archivos de logs o resulta demasiado tedioso ir configurando todos, incluso aunque utilices sistemas automatizados como Puppet.

De ahí la necesidad de crear un pequeño script de bash, más radical (bajo responsabilidad del usuario) que previo respaldo de todos los logs en el home, borre todos los archivos de log más antiguos que una determinada fecha dada en días y tambien borre archivos rotados con logrotate.

Lo he llamado zerolog y este es código que realiza esas tareas:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

#!/usr/bin/env bash
 
#    zerolog is a bash script for delete common logs on /var/log and
#    avoid fill the disk with rotated files. It backups previously
#    the logs on home folder and it can preserve logs older than
#    a date given on days. This is useful for avoid slow sessions
#    starts on bash terminals.
 
#    Copyright (C) 2012 by Ángel Guzmán Maeso, shakaran at gmail dot com
 
#    This program is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation; either version 3 of the License, or
#    (at your option) any later version.
 
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#    GNU General Public License for more details.
 
#    You should have received a copy of the GNU General Public License
#    along with this program; if not, write to the Free Software
#    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
usage(){
    echo 'Usage: $0 [DAYS]'
    echo 'DAYS: Remove logs older than DAYS. Default: 0 days'
    exit 1
}
 
if [ `/usr/bin/id -u` -ne 0 ]; then
    echo 'Please, run this script as root'
    usage
fi
 
if [ -z "$1" ]; then
    MTIME_DAYS=''
else
    MTIME_DAYS='-mtime +$1'
fi
 
echo Currently logs size: `du -hs /var/log/ | awk {'print $1'}`
echo 'Backup the log folder on home'
tar zcPf ~/logs-`date '+%m-%d-%y-%H:%M:%S'`.tar.gz /var/log
 
echo 'Removing rotated log files'
find /var/log/ -type f $MTIME_DAYS -name *.gz -exec rm -rf '{}' \;
 
echo 'Emptying the logs'
for logs in `find /var/log $MTIME_DAYS -type f`; do > $logs; done
 
echo Currently logs size: `du -hs /var/log/ | awk {'print $1'}`

El script esta subido a github en:

https://github.com/shakaran/scripts/blob/master/zerolog.sh

Por si alguien quiere hacer fork, modificarlo o en un futuro hago nuevas modificaciones. Esta licenciado bajo GPLv3.

Ejemplos de uso

Para usarlo, requiere ser administrador, ya que no todos los logs pueden borrarse como usuario regular.

Puede usarse como:

1

$ sudo ./zerolog.sh

Que borrará todos los logs rotados y vaciara existentes, ademas de crear un backup previo en el home del tipo logs-<FECHA>.tar.gz con la fecha actual de la ejecución. Por si quieres conservar los logs o decides restaurarlos por algún motivo.

Si quieres borrar sólo los más antiguos de X días, por ejemplo los más antiguos de hace una semana:

1

$ sudo ./zerolog.sh 7

Incluso puedes crearte un alias zlog:

1

$ alias zlog="sudo sh /ruta/zerolog.sh"

También puedes añadir el script como una tarea cron y que se ejecute a diario, por ejemplo:

1

$ sudo crontab -e

Y añadir:

1

0 0 * * * sh /ruta/zerolog.sh > /dev/null 2>&1

Por otro lado, si te importan poco tus logs o quieren guardar solo los más recientes, puedes optar por una opción más drástica como:

1

0 0 * * * find /var/log/ -type f -mtime +7 -exec rm -rf '{}' \; > /dev/null 2>&1

ZeroC Ice logo

Ice o  Internet Communications Engine  es una plataforma middleware orientada a objetos, que permite escribir aplicaciones distribuidas con muy poco coste. Provee herramientas, APIs, bibliotecas y bindings para la construcciones de aplicaciones cliente-servidor de forma escalable.

Es una buena herramienta de trabajo para entornos que sean bastante heterogéneos, donde el cliente y servidor puede estar escritos en lenguajes de programación muy diferentes y ejecutarse sobre plataformas, sistemas operativos o arquitecturas muy variadas.

Su fuerte radica en un gran soporte de tecnologías de red con interacciones a bajo nivel de red, enfoque en la lógica de las aplicaciones y portabilidad de entornos muy diversos.

Otro punto fuerte a destacar es la abstracción de red para detalles como abrir conexiones de red, serializado y des-serializado de datos transmitidos por red, reintentos fallidos de conexión y muchos otros detalles más de bajo nivel que ayudan a simplificar la capa de red.

Soporta bindings con bastantes lenguajes de programación como: 
C++, .NET, Java,Python, Objective-C, Ruby, PHP, y ActionScript.

Debido a estas características Ice es usado por muchas compañias con misiones críticas en el mundo. La curva de aprendizaje es sencilla y existe una gran infraestructura de aplicaciones técnicas que se demandan.

Ice es software libre bajo licencia GPLv2 y comercializado bajo la marca ZeroC.

Instalación con repositorio

Esta opción de instalación es sencilla y cómoda, pero no esta aconsejada para desarrolladores que necesiten actualizaciones con bastante frecuencia o disponer de una versión determinada.

Descargar el repo (en estos momentos de escribir el artículo para la versión 3.4) desde la página web oficial:

1

 # wget http://download.zeroc.com/Ice/3.4/rhel6/zeroc-ice-rhel6.repo -O /etc/yum.repos.d/zeroc-ice-rhel6.repo

Activar el repositorio e instalar ZeroC Ice y biblioteca Ice para Python:

1

 # yum --enablerepo zeroc-ice install -y ice ice-python

Instalación desde fuentes

Si tu necesidad esta más orientada al enfoque desarrollador, puedes compilar la versión que requieras o en desarrollo. En el momento de escribir este artículo se compila para la versión 3.4.2 de Ice.

Instalar portable C preprocessor desde el repo de ZeroC Ice (también puedes optar por bajar y compilar los fuentes de mcpp, pero no es necesario):

1

# yum install -y mcpp-devel

Después bajamos los fuentes de Ice, descomprimimos y compilamos la versión en C++ y su binding para Python (en este caso para la versión 3.4.2 de Ice). Puedes hacerlo todo en una línea con:

1

# wget http://zeroc.com/download/Ice/3.4/Ice-3.4.2.tar.gz; tar xvzf Ice-*.tar.gz; cd Ice-*/cpp; make; make install; cd ../py/; make;make install

La instalación de Ice quedará bajo /opt/Ice-3.4.2/ y el binding python sobre /opt/Ice-3.4.2/python. Es importante recalcar que el binding python se asociara con la versión de defecto de Python en el sistema si dispones de varias.

Por último, es necesario indicar el el PATH y el PYTHONPATH del sistema, donde se encuentra la instalación de Ice y su binding de Python:

1
2

# export PATH=/opt/Ice-3.4.2/bin:$PATH
# export PYTHONPATH=/opt/Ice-3.4.2/python:$PYTHONPATH