SFourmi on the web using Emscripten

The SFourmi project is now running directly in the web browser, being compiled from C++ into WebAssembly using Emscripten.

Try it here

but be patient, it is very slow to load (explanations below)

You can then use mouse and keys (F2, click and select, and so on)

Why ?

For my new job I wanted to explore the possibility of running C++ code in the browser. I decided to use SFourmi as a testing project, because it only uses libraries supported by emscripten (SDL), and the project can be shown to anyone with a web browser !

Build it

You need to have emscripten installed on your system, follow the instructions here

Don’t forget to run the scripts to get all emscripten tools in the path

source emscripten-install/emsdk_env.sh

Get the source code of SuperFourmi using git from https://github.com/manikbh/SFourmi/

cd SFourmi
mkdir build
cd build
emcmake cmake ../
emmake make
cd src
emrun SFourmi.html # To run it

How?

I found the documentation of Emscripten lacking, and the error reporting really bad, so for a long time it did not work and I had no idea why, even though I followed multiple tutorials that seems contradictory or just outdated as emscripten evolves.

I first added CMake support to SFourmi to reorganize the way the code is compiled, as old autotools are now quite deprecated. As I was learning CMake, the CMakeLists.txt are not very clean, especially as target dependencies do not seems to be carried over correctly when using emscripten: I had to repeat the SDL library dependency on multiple targets even though it should be inherited and works well when using normal g++-based compilation.

Then I noticed a number of badly programmed parts that had to be corrected. I removed the way to write logs to a file so that it would be displayed in the web console (that’s slow, and currently it is compiled in debug mode so the log is very verbose).

I added in CMakeLists.txt

if( ${CMAKE_SYSTEM_NAME} MATCHES “Emscripten”)

set(USE_FLAGS “-s USE_SDL=2 -s USE_SDL_IMAGE=2 -s USE_SDL_TTF=2 -s SDL2_IMAGE_FORMATS='[\”xpm\”]’ -sALLOW_MEMORY_GROWTH –embed-file ../images”)

set(CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS} ${USE_FLAGS})

set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} ${USE_FLAGS})
set(CMAKE_EXECUTABLE_SUFFIX .html)

else()

find_package(SDL2 REQUIRED)

find_package(SDL2_image REQUIRED)

find_package(SDL2_ttf REQUIRED)

endif()

What is important there is that SFourmi uses SDL SDL-image and SDL-TTF. For SDL-image, there was no png format at the time to load the background and sprites, so everything was stored in XPM non-compressed format because this format can be just included in a C++ file, so you get a variable containing the image data and you don’t need to distribute the image file with the software as it is embedded in it.

Nowadays we should use PNG to reduce the size of the file and use SDL-image’s PNG support to use it.

However, I tried to have it work as it is, so I added -s USE_SDL=2 -s USE_SDL_IMAGE=2 -s USE_SDL_TTF=2 flags, but I never saw anything.

Not being sure what was wrong, I followed different examples of using SDL2 with emscripten. Some of those example explain that you have to modify the main function of the program, some said it was not necessary. I finally modified main function with #ifdef for EMSCRIPTEN, but that did not solve the problem.

There were still two main issues: bad error condition handling in the code when not finding a font to load or failing to read the main image. I added error handling code, and saw that the image was not loaded. How could it be, it was embedded as XPM in the source code ! In fact you have to add another flag when using SDL-image to declare the file formats you are using, otherwise it will just fail to load images without any error message. -s SDL2_IMAGE_FORMATS='[\”xpm\”]’ was the flag to add !

Unfortunately, SFourmi also needs a font file to display text, and it is loaded from a directory. But that cannot work from the web browser, so you need to create a virtual filesystem containing all files that will be accessible from the software, in this case the images directory, and use relative paths. Fortunately there is just another flag to add: –embed-file ../images

Looking in the web console, I saw another error about lacking memory, with another flag to use to solve the problem: -sALLOW_MEMORY_GROWTH

And finally it worked !

Todo list

  • Cleanup CMake files and Emscripten flags
  • Use PNG images to compress the huge XPM file
    Unfortunately it seems that just adding png to the list of formats in the SDL2_IMAGE_FORMATS flag does not work, SFourmi just shows a black screen again. Maybe it needs to manually add libpng flags to the build ?
  • Cleanup which keys do what, allow parameters other than default map (used to load an ini file, should find a way to provide simulation parameters – e.g. terrain size, number of ants and families)
  • Display on the compiled SFourmi.html web page the usage instructions and remove Emscripten huge logo.
  • Add a CMake install target so all required files go into one clean directory.

 

 

Posted in software project | Tagged | Leave a comment

(Français) Plastiphages

Sorry, this entry is only available in Français.

Posted in Misc. | Leave a comment

CARMEN collective – Consciousness, Attention and MENtal Representation

CARMEN

Since 2019 I have been involved with the CARMEN collective in Grenoble, where scientists from multiple fields (ethology, philosophy, neuroscience, medicine, biology, mathematical modeling and engineering) discuss on topics related to consciousness.

Conference

In december 2020 a day of presentations and discussions was organized (in french), titled “La conscience dans tous ses états“.

All videos (audio and image quality are low, unfortunately) are available on youtube.

The last part of the day was a discussion on consciousness in animals and machines, stemming from a questionnaire that was submitted to people registering for the conference and their relatives and friends. Two students presented the results and Christian Graff and myself discussed them with them and with the audience.

Book

The collective has decided to write a book (in french) on its discussions about consciousness.

It will be submitted to the editor in 2021.
I wrote jointly with Michel Dojat and Christian Graff the chapter 8. After proofreading by the other members of the CARMEN collective the chapter’s preprint was put online on ResearchGate.

A short analysis of the questionnaire for the conference will follow, and I hope to write an english translation of the chapter. EDIT: english version published as preprint here !

Posted in Consciousness | Leave a comment

SFourmi – Super Ant

SuperFourmi (SuperAnt)

a multi-agent ant simulator from the last millenium (!) – Github link

History

It was the first C++ project I developped in highschool (Michel Tran Ngoc, Manik Bhattacharjee for conception and code, graphics by Tristan Millner and some code refactoring by François Leiber) – between 1998 and 2002.

Its old website is still online, last updated 18 years ago !

We did read books about ants from the national library, observe ants in nature and take pictures.

 

 

 

 

The project was a learning opportunity for object-oriented programming in C++, graphics programming in Windows and Linux, using DirectX, GTK and finally SDL.

Basic explanation

Ants explore their 2D world in search of food that they bring back to the anthill.

Each ant independently assesses its situation and environment, and according to its class (worker, queen, nurse, warrior) does the appropriate task. They leave pheromone trails (blue arrows for food tracks, red dots for danger), move on the map, get food (yellow patches) and breed new ants (from larva) underground in their anthill. They can kill other insects and ants from other anthills (small colored dots for dead insects).

The interface shows both a minimap with all ants (on the left), and a local view (right). An ant can be selected to see its characteristics (on the left) and the ant’s image changes when it is carrying food.

Worker ants retrieving food and leaving/following pheromone tracksWarrior ant next to a dead insect and danger pheromones Underground, nursing ants feed the larva next to the queen while workers bring food in

Current status

I recently (february 2020) tried to make it run again, and I was able to compile it using old librairies still distributed on linux.

The latest 2D version was compiled after some code tweaks (it turns out that 64 bits computing and badly coded C++ 98 don’t work together that well). It runs pretty well, even though there is no hardware acceleration for the 2D display.


Cleaning up the source code would be a lot of work, but it would be nice to see it running with newer libraries such as SDL2 with hardware acceleration, with a clean C++-20 code and taking advantage of parallel processes through multithreading (or even GPUs) !

The 3D version that Michel developped in the last year does compile but crashes and does not seem to work.

All source code was uploaded to Github, the new licence is GNU Affero GPL v3.

Posted in software project | 2 Comments

Tracing an old train track in OpenStreetMap’s Umap

From 1901 to the beginning of the 1950’s there was a train line between the french village of Egreville and the larger city Sens. Because I have some with a village on the track I wanted to learn more about it. The tracks were removed but amateurs gathered a lot of data about the surviving buildings and the path. I was also able to use satellite and aerial images (IGN for historical and recent ones, Bing Maps and Google Maps).

This is the result of this work, but there might still be some mistakes.

See full screen

See full screen

Sources :

http://www.inventaires-ferroviaires.fr/hd77/77168.a.pdf
– Google maps, Bing maps with some traces (limits of fields and dark shapes in the fields marking the old track)
– historical and modern aerial pictures and maps from french IGN https://remonterletemps.ign.fr/comparer/basic?x=3.257303&y=48.221596&z=15&layer1=ORTHOIMAGERY.ORTHOPHOTOS.1950-1965&layer2=GEOGRAPHICALGRIDSYSTEMS.MAPS.SCAN50.1950&mode=doubleMap
– Data from online forum http://www.passion-metrique.net/forums/viewtopic.php?f=2&t=7647&hilit=CFY&start=45

Posted in Misc. | Leave a comment

setGID, ldd and library not found

Today I was bewildered by a very unusual problem. Maybe this will help someone else with the same problem !

I was running an executable as myself and it was working perfectly. I had to run it as root to access some protected directories, so I used sudo:

sudo myprogram

I got

error while loading shared libraries: libsomething.so: cannot open shared object file: No such file or directory

Ok, this must be because sudo won’t use the LD_LIBRARY_PATH that I defined, so I try:

sudo -E myprogram

Same error… Ok, I will try to check what’s going on as root, with ldd.

sudo bash
export LD_LIBRARY_PATH=/path/to/my/lib/
ldd myprogram
      libsomething.so  -> /path/to/my/lib/libsomething.so
# Ok ! Everything works as expected, I can run the program
./myprogram
error while loading shared libraries: libsomething.so: cannot open shared object file: No such file or directory

ldd finds all necessary libraries, but the executable still fails to run because it cannot find a library ??

Ok, maybe I don’t have the rights to read or execute the library or the directories in its path ?

chmod -R a+rX /mydirectory does not change anything, even though any user can now access the libsomething.so file.

After a LOT of searching around, I found an explanation : if your executable or library is setuid or setgid, because of security implications, LD_LIBRARY_PATH won’t be used !

Finally, I did a find /mydirectory -type f -exec chmod g-s {} \; to remove the setGID bit on all files (but keeping it on directories). SetGid bit was set to force new files to  belong to the directory’s group : it was only necessary to setgid the directories, not the files !

 

 

 

Posted in TechTips | Leave a comment

Virtual tour of Louis-le-Grand high school (1996-1998)

Project LLG

I decided to dig up an old project, a virtual tour of my highschool in 1997-1998 and to convert it to a html5/javascript/css web project to learn more about these languages.
At the beginning, this was about visiting the highschool from picture to picture in a Visual Basic 6 software.facadeDeCote

Project team

  • Idea, conception & programming: Manik Bhattacharjee
  • photo : Manik Bhattacharjee and Michel Tran Ngoc
  • GCD compressed image format and C library : Laurent Demonet
  • icons, 3D modeling of a classroom : Tristan Millner

Making of

Pictures were taken with analog manual SLR cameras, digitized using a flatbed scanner and converted to GCD by myself on a Pentium 133 MHz computer. I then used a small program to define click zones for each image. A few pictures were taken using a low-res digital camera lended by a friend (Habib Shoukry), a Sony that stored the pictures directly on a floppy disk.

Tristan Millner created the UI design, some icons and a 3D model of a classroom (I just have a screenshot left of that). 3D modeling was done to create a 3D game, doom-style, in our highschool. But the project stopped there…

We were given access to the scientific museum and the old library, but I did not find those pictures yet.

HTML5 conversion

This project dating from 17 years, I lost some data in the mean time. I don’t have the digitized JPEG files (deleted probably because the hard drive was full, and maybe on some old CD-R somewhere), the original film is probably in a cellar and I still miss a number of pictures in the version I recovered. If I find better data on old CD-R or ZIP disks, I will update the online version.

En 1997, Visual Basic could not display JPEG images, only BMP. But those files needed so much space that it was not possible to store all files in BMP and having the whole project on one CD. Laurent Demonet offered to design a file format and C library using similar principles as GIF, with an OCX element to be inserted in Visual Basic UI. Image quality seemed ok at the time (reduced color palette and resolution sufficient at the time).
Fortunately I found the source code of the OCX control, so I wrote a C++ program to convert the old GCD files back to JPEG (thanks to Qt library).

The labels of images were saved in simple text files, but character encoding was windows 95 french encoding – CP1252. I thus converted these to UTF8 using iconv.

I did not recreate the original UI, but just wrote a simple HTML5 page using AJAX to get images and click zone data.

I still haven’t put the small highschool map with active points to start on different places… yet !

Posted in software project | Leave a comment

Ubuntu 15.04 : Adding a ppa behind a proxy

To install a new PPA (for example the KDE backports to have the latest stable KDE on Kubuntu 15.04) on a connection using a proxy for http/https access :

export http_proxy=http://myProxy:port
export https_proxy=http://myProxy:port
sudo -E add-apt-repository ppa:kubuntu-ppa/backport

The “-E” allows sudo to keep the environment variables defined for the proxy access.

To get apt to use the proxy, create a file /etc/apt/apt.conf.d/50proxy :
Acquire::http::Proxy "http://myProxy:port";
Acquire::https::Proxy "https://myProxy:port";

Posted in TechTips | Leave a comment

Gitlab behind a reverse proxy with SSL termination

I needed to install a git server with a web interface. I decided to install Gitlab Community Edition. But I had some trouble using it behind a reverse proxy that does SSL termination

  • I added a new virtual machine (VM) called GIT to my virtual machines server SERV. It contains a Debian minimal installation.
  • I installed the debian package Omnibus Gitlab 7.9 in GIT.
  • The VM GIT is connected to a (virtual) private network which allows SERV to access all its VMs.
  • Using iptables, SERV redirects port 1212 to port 22 of GIT. This gives me SSH access to GIT. In /etc/iptables/rules.v4
    -A PREROUTING -i eth1 -p tcp –dport 1212 -j DNAT –to-destination 172.16.0.5:22 -m comment –comment “Redirect port 1212 to GIT”
  • SERV serves multiple websites using a number of DNS names. For example, mysite.eu and mygit.eu will redirect to SERV’s IP address.
  • SERV runs NGINX as a reverse proxy with SSL termination. This just means that each request to https://mygit.eu is received by nginx which decrypts the https request and forwards it to GIT as an HTTP request. Thus, the outside address is HTTPS but from the viewpoint of gitlab, the coming request is HTTP. In /etc/nginx/sites-enabled/default for each server :
    server {
       listen 443;
       root /var/www/;
      server_name mygit.eu;
      index index.html;
      include /etc/nginx/include/basicsslconfig;
      ssl_certificate /etc/nginx/httpskeys/mygit.eu.pem;
      ssl_certificate_key /etc/nginx/httpskeys/mygit.eu.key;
      location / {
        proxy_set_header Host mygit.eu;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto http;
        proxy_set_header X-Forwarded-Ssl on;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://172.16.0.5:80/;
      }
    }
  • To access gitlab website, I just use https://mygit.eu
  • SSH access for git : to get this to work, I just set in /etc/gitlab/gitlab.rb gitlab_rails[‘gitlab_shell_ssh_port’] = 1212
    and the SSH URL displayed for a repository is ok and works.
  • HTTP access for git : if I set
    external_url ‘http://mygit.eu’
    The lack of https means that the url won’t work.
    But if i use https, gitlab will configure its own nginx as https server and the forwarded http request won’t work.

Workaround : the reverse proxy replies to http requests by redirecting to the https address. So even if the wrong (http) address is displayed, it will still work. In /etc/nginx/sites-enabled/default

server {
listen 80;
server_name mygit.eu;
# Temporary redirect while keeping the same request -> 307. Permanent redirect -> 301
return 301 https://mygit.eu$request_uri;
}

Real solution : there is a new setting (still not available in the debian package) to allow for this according to https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/nginx.md
nginx[‘listen_https’] = false

This should force the GIT nginx server to accept http and not https requests, even if external_url contains https.

Posted in TechTips | Leave a comment

Sciences en Marche 2 : Grenoble-Lyon

On october 4th 2014, Sciences en Marche militants from Grenoble went biking from Grenoble to Lyon.

Sciences en marche – Grenoble -> Lyon par m_b_38
Here is a map of what I did :
– In red, Grenoble-Rives on my bike. The bike was sent back to Grenoble on a train with a colleague.
– In blue, the sweep vehicle that started long after the bikers gets lost then goes to Eclose to meet the bikers
– In red, the sweep vehicle follows the bikers to Lyon’s ring road. The bikers ride alone to the city center, avoiding the ring road. The car parks near the river.
– In green, going on foot to the meeting point.

Link to the map

Same map (but small)

Voir en plein écran

Posted in Politics & society | Leave a comment