DataBackbone
From OpenTom
Contents |
What
This project will provide an open interface for programmers to register callback functions in their own projects so that these are triggered when action happen on file handles.
Why
Without the SDK there is barely a good way to access e.g. NMEA-data, since TTs navcore holds the filehandle "/dev/gpsdata" and does not let anyone else access it.
From TTs point of view, that is totally ok, since the system is not meant to be "open".
So programmers have to "hack" the system to get access, if they want everything, in the above case one set of coordinates every second, without the navigation to freeze.
Background
I (Joghurt) talked to several programmers about this, first Roussillat (long time ago), then with Escor and lately with Amacri, because even if we used a hack so we can access files parallel (kind of), these hacks sometimes block each other. :(
That's why I finally start this one project, so we can altogether code an interface, that can do everything we need.
How
I thought about a shared library (.so) as base mechanism that is connected to ttn via LD_PRELOAD, like my new project 'Height' and Escors 'TTMPlayer' (which sadly do NOT yet work together :( ).
I thought about freely accessible functions like "Call my function 'foo' when there is action 'bar' on file 'baz'" where 'bar' is like "open", "close", "read", "write" or "ioctl" and so forth. Also a function for deregistering.
Maybe also functions for manipulating data like "Do not send this data to TTs navcore", so we can get a grip on touchscreen events and stuff.
Discussion
Is it really necessary?
Bello: I do not understand, why this is neccesary. When opeing a file you can specify if it should be locked or not. You also can duplicate a file handle etc... A lot of io-functions are available.
Joghurt: Yes, but can you access a file (e.g. "/dev/gpsdata") if TomTom opens and never closes it? (yes, why not?) And if you can, you will nevertheless not be able to get data parallel to TTs navcore, you will always read concurrent to them, what they read you can't read and vice versa. Please correct me if I'm wrong.(Hae?) I mean, /dev/gpsdata provides data only once, and when it's read, it's gone. As far as I've understood...Hm, but each application gets the data starting from the time when it opened the device-file. So data, which has been read by one application can still be read by the other. At least with firmware versions up to 8.3 this works like this. Of course you can write the device-driver such, that only a single application is allowed to open it, bus as far as I know the /dev/gpsdata driver is not written like this up to now. It is a normal charackter device with at least some kBytes of internal buffer. You can give it a try with a cat /dev/gpsdata from a shell. This should work independantly of how many other applications (ttn, TTTracklog et cetera) have already opened it.
Amacri: IMHO, ttn constantly reads from /dev/gpsdata (e.g., through a select() or something like it) and, in case of read failure, it reopens the device; when issuing a cat /dev/gpsdata from a shell, ttn fails and cat outputs data instead of NavCore until ttn reopens the device. At least, I think so...This can only be, if the shell makes the ttn application stop or suspend. In this case, when it does not read the stuff from the device, the buffer might get filled up and this can cause a read error, when ttn is resumed/the shell quits. This is a guess. Maybe someone simply can do some real tests, and then we know. Also I have the suspicion, that the behaviour might be different with different firmware versions. With 7.x I can say, that ttn and TTTracklog (which permanently reads all GPS data from open /dev/gpsdata in the background) do not at all interfer with each other.
Amacri: Suppose two applications concurrently read from /dev/gpsdata; example:
cat /dev/gpsdata > /mnt/sdcard/DumpTty1 &
cat /dev/gpsdata > /mnt/sdcard/DumpTty2 &
do you think that both applications read the same data, or that, in other terms, /dev/gpsdata is capable to automatically duplicate data to each reading application so that all receive the same information? Example, do both /mnt/sdcard/DumpTty1 and /mnt/sdcard/DumpTty2 have the same content? I would not expect so... Besides, Hammerhead GPS devices require reading from /var/run/gpspipe, which is a named pipe; do you think that also a named pipe is able to perform this data duplication in case of concurrent access, or it will be another race condition issue?
And you won't need to loop and check if something changed, you will get informed if something happens, so that way your tool would consume as little performance as possible. (Hae? We have the select() function, which can do this.) Ok, you're right. :)
And you could do special tricks, like modifying the Framebuffer before it is displayed, like I do with 'Height' using ioctl. I don't know another easy way to do something like that. And this way, anyone could do it, and at the same time!
amacri: in fact I am also thinking to implement this interface with Event_Logger (an alpha version of this interface is already running) and agree that a shared approach would be appropriate. Ideal would be to run Event_Logger 8.2, Height, TTMPlayer and other programs concurrently, with easy installation for users as well as manageable coding from each programmer.
Bello:I still do not understand what you are talking about. Having a file opened twice (and get the same data on many different applications) is not at all a problem. Except for something has canged in a newer Firmware version (please tell me if this is the case), but I can hardly imagine that the principles have changed, because it is still linux, isn't it?.
Joghurt: I admit that I need some private lessons in C/C++ since I'm not that good in these languages... ;) But as far as I understood, you still can't intercept ioctls, and you can't modify, inject or drop data with your approach. Please correct me if I'm wrong! And with an open interface, we would no longer have the problem of how to start daemons (since cleanup.txt no longer seems to work) and handle concurrent ttn-files or lost ones on bootloader-update and so on...
Bello:To my experience /dev/gpsdata does not work before(!) ttn has started. This may come because it gets configured by the ttn application with some ioctl command. With your code injection mechanism, you can find out, which ioctls are used on /dev/gpsdata on startup of ttn. By the way: code injection using the dynamic linker works also with more than one library. But can several Libraries overwrite the same function? I have tested this: definitly YES! The functions are overloaded in the order the libraries are loaded. One needs to make sure, that the overloaded (and therefor hidden) functions get called by the overloading ones in the correct order, but the dynamical linker library has functions which allow to adress all (the first and the last) hidden versions of these functions. So each application could bring their own modifications, with little interference on each other if programmed well. To catch the ttn application itself, the ttn-script has to be modifyed. Doing this is a modular way is a bit tricky. But one can invent a chaining mechanism: Inject some lines at the beginning of the file, which can be autoremoved on deinstallation. Or: agree on a common place (e.g. the prelib folder) where to put all libraries, which should be used for overloading. Code injection is not the best way to program. The stuff gets spagetti-code like, and should be avoided, whenever possible.
Order of overloades symbols in dynamically loaded libraries (See RTLD_NEXT):
dlsym
The function dlsym() takes a "handle" of a dynamic library returned by dlopen and the NUL-terminated symbol
name, returning the address where that symbol is loaded into memory. If the symbol is not found, in the speci-
fied library or any of the libraries that were automatically loaded by dlopen() when that library was loaded,
dlsym() returns NULL. (The search performed by dlsym() is breadth first through the dependency tree of these
libraries.) Since the value of the symbol could actually be NULL (so that a NULL return from dlsym() need not
indicate an error), the correct way to test for an error is to call dlerror() to clear any old error condi-
tions, then call dlsym(), and then call dlerror() again, saving its return value into a variable, and check
whether this saved value is not NULL.
There are two special pseudo-handles, RTLD_DEFAULT and RTLD_NEXT. The former will find the first occurrence of
the desired symbol using the default library search order. The latter will find the next occurrence of a func-
tion in the search order after the current library. This allows one to provide a wrapper around a function in
another shared library.
Multiple libraries to be loaded:
LD_PRELOAD
A whitespace-separated list of additional, user-specified, ELF shared libraries to be loaded before all
others. This can be used to selectively override functions in other shared libraries. For
setuid/setgid ELF binaries, only libraries in the standard search directories that are also setuid will
be loaded.
Example code, to overload the fopen function, to find out which files are opened:
static FILE *(*old_fopen)(const char *path, const char *mode) = NULL;
FILE *fopen(const char *path, const char *mode) {
printf("FOPEN: <%s>\n",path);
if(old_fopen==NULL) {
printf("Installing wrapper:\n");
old_fopen=dlsym(RTLD_NEXT,"fopen");
printf("old_fopen = %p\n", old_fopen);
}
return(old_fopen(path,mode));
}
Important is, that you call the old function at the end, so that other functions (overloaded by other applications) get a chance to be called.
Roussillat (Tripmaster's author) : I'm trying to hook mmap() and memcpy(), but I don't succeed to find the right code for this, can you help me ?
Standards:
Maybe it is just a matter of defining some good standards to follow for all add-on applications, so that they better work together. First, the directories should be standardized. Good practice would be:
/mnt/sdcard/bin all excecutable programs go here, this directory will be added to PATH /mnt/sdcard/bootup directory with executables, replacement for the no-longer-working cleanup.txt ...? /mnt/sdcard/suspend directory with executables, fired on suspend ...? /mnt/sdcard/resume directory with executables, fired on resume ...? /mnt/sdcard/lib all shared libraries on which the applications depend come here, and this will be added to LD_LIBRARY_PATH /mnt/sdcard/prelib all shared libraries which should be used for overloading come here, and these will be added to LD_PRELOAD /mnt/sdcard/runonce directory with executables, that are deleted after execution, for postinstallation or uninstall - What do you think? /mnt/sdcard/etc all configuration files go here. /mnt/sdcard This directory will be made the HOME directory (hm, not so clever, other suggestions are welcomned) /mnt/sdcard This directory will be the PWD for all applications when called (hm, also not so good)
We should not solely consider /mnt/sdcard but also /mnt/movinand ...
Joghurt: If this library-over-library-overloading works like intended, it's fine with me. :)
Bello: I would say: just give it a try.
Bello: What is movinand?
Joghurt: If your TT has internal mem and an sd- or microsd-card, internal is "sdcard" and sdcard is "movinand". ;)
Hmm, most of this stuff can be done by a single ttn-file, if that library-overloading works... Maybe like this?
#! /bin/sh export PATH=/mnt/movinand/bin:/mnt/sdcard/bin:$PATH export LD_LIBRARY_PATH=/mnt/movinand/lib:/mnt/sdcard/lib:$LD_LIBRARY_PATH for i in /mnt/movinand/runonce/* /mnt/sdcard/runonce/*; do (($i; rm $i)&); done ls -1df /mnt/movinand/suspend/* /mnt/sdcard/suspend/* >>/etc/rc.suspend 2>/dev/null ls -1df /mnt/movinand/resume/* /mnt/sdcard/resume/* >>/etc/rc.resume 2>/dev/null for i in /mnt/movinand/bootup/* /mnt/sdcard/bootup/*; do ($i &); done ls -1df /mnt/movinand/prelib/*.so /mnt/sdcard/prelib/*.so >/etc/ld.so.preload 2>/dev/null ttn
amacri: Interesting discussion.
Do you think that when two (or more) programs concurrently read data from /dev/gpsdata, all of them get the same duplicate information? YES. You have to see it like this: In the kernel there is a buffer, which always holds the latest say 4kBytes of the datastream from the GPS device. When an Application opens the device It has access to this buffer and can read from the beginning of the buffer to the end of it, in which case the read command waits for more data to arrive. You will really get everything the GPS emmits. This is true for each appplication. When the device is closed, the application gets decouppled from the datastream and until it opens it again some data might get lost. This is a proplem for script-applications, because it is difficult to keep the device/file opend for the whole time the script is running. a 'cat /dev/gpsdata' (used in a script) opens, reads (until no data is available any more at that time), and immediately closes the the device, so data can get lost. amacri: Yes, but what happens if the same special file (serial tty) is opened by two applications at the same time? I doubt that both receive the same information; I think instead that the latter (the last one attached to the input tty) stoles data to the firmer: they will both contend the resource and I am testing this with TomTom (at least, I believe). As soon as an external application opens /dev/gpsdata, NavCore (which constantly receives data from that special file) seems to realize of that, and might desynchronize for some instants with the road path (that was the issue that I wished to overcome). I also think that this does not happen when using LD_PRELOAD. Anyway, I am very interested about your opinion; I would ask you to better detail why this might not be true, thanks.
I would think that both contend information from that same source. In my opinion LD_PRELOAD would be a possible way (easily implementable) to overcome this device contention, allowing both programs (e.g., ttn and an external application) to read exactly the same data coming from the same GPS device, without any concurrence issue. Anyway, external programs invoked by ttn will all have this variable exported (maybe unsetenv("LD_PRELOAD") should be called within the wrapper constructor).
By the way, I tested a whitespace-separated list of shared libraries in LD_PRELOAD and unfortunately it does not seem to work (ttn crashes).
Non working example:
LD_PRELOAD='/mnt/sdcard/Height/height.so /mnt/sdcard/Tracer/devshare.so' /bin/ttn &
Hm, this is a surprise. check, with env, what is really in the environment var. If ttn crashes, this would mean, that the linker has done something. So maybe there is another problem, which should be analysed...(Without the sourcecode I cannot help you here...)
amacri: Also my surprise; pherhaps wrapping the same system call cannot be nested by subsequent shared libraries loaded through LD_PRELOAD; I am absolutely unsure of this.You are not right. See tests below.
I do not think that wrapping system calls using LD_LIBRARY_PATH or /etc/ld.so.preload (good for development and testing) would be the best solution (there's the potential problem of influencing other programs even if only one – namely “ttn” – shall be affected). Unfortunately I was not able to make /etc/ld.so.preload work (e.g., echo '/mnt/sdcard/Height/height.so /mnt/sdcard/Tracer/devshare.so' >/etc/ld.so.preload)
Finally, from my point of view, standard directories might be simplified.
Testing goes on.
Tests
Bello: Tested using two .so libraries, wrapping the same function:
- Whitespace separated list in LD_PRELOAD='./libwrap_it.so ./libwrap_it2.so' testprog
- Both libraries are preloaded by the dyn-liker
- It works, both functions get called one after the other, just as expected. The function from libwrap_it.so get called first, then it called the function from libwrap_it2.so and finally the original function.
Joghurt: Hmm, maybe you can show us your test code...? My testing ended in the same results as amacris, the system just hangs with my two test-libraries. :(
Bello: here it is:
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <signal.h>
#include <execinfo.h>
static char *(*old_strcpy)(char *dest, const char *src) = NULL;
char *strcpy(char *dest, const char *src) {
char *p=dest;
printf("STRCPY from library Nr 1: <%s>\n",src);
if(old_strcpy==NULL) {
printf("Installing wrapper:\n");
old_strcpy=dlsym(RTLD_NEXT,"strcpy");
printf("old_strcpy = %p\n", old_strcpy);
}
return(old_strcpy(dest,src));
}
Now compile this and name it strcpy1.so. Then change the text in the printf-statement, compile it again and call it strcpy2.so. Just for info, the compile command is:
$(CC) -fPIC -shared -ldl -o strcpy.so wrap_it.c
Now use the libraries (from TTconsole):
LD_PRELOAD='./strcpy1.so ./strcpy2.so' xbasic
The output looks like:
> >STRCPY1: < >STRCPY2: < STRCPY1: <> STRCPY2: <> STRCPY1: <> STRCPY2: <> STRCPY1: <> STRCPY2: <> STRCPY1: <> STRCPY2: <> STRCPY1: <> STRCPY2: <> STRCPY1: <B> STRCPY2: <B> STRCPY1: <> STRCPY2: <> STRCPY1: <H> STRCPY2: <H> STRCPY1: <D> STRCPY2: <D> STRCPY1: <C> STRCPY2: <C> STRCPY1: <> ...
Joghurt: Haven't tried it on a TT yet, but on my local Linux it works. :)
Looks like you need to fetch the pointer to the "old" function in the new function, and not in the constructor of the library, like I did...
And the variable to store the function pointers HAVE TO be static, otherwise you get a loop... :(
Update: Nope, just need to make the function pointers static, you can get 'em in the constructor without problems. Hell knows why... :)
My class is:
#define _GNU_SOURCE #include <stdio.h> #include <dlfcn.h> #include <stdlib.h>
static int ( *org_open )( const char *file, int oflag, ... ); static ssize_t ( *org_write )( int fd, const void *buf, size_t nbytes ); static ssize_t ( *org_read )( int fd, void *buf, size_t nbytes ); static int ( *org_close )( int fd );
void __attribute__( ( constructor ) ) strcpyXXX_init( void ) {
printf( "XXX loaded\n" );
org_open = dlsym( RTLD_NEXT, "open" );
org_write = dlsym( RTLD_NEXT, "write" );
org_read = dlsym( RTLD_NEXT, "read" );
org_close = dlsym( RTLD_NEXT, "close" );
}
void __attribute__( ( destructor ) ) strcpyXXX_uninit( void ) {
printf( "XXX unloaded\n" );
}
int open( const char *file, int oflag, ... ) {
printf( "XXX: open()\n");
return org_open( file, oflag );
}
ssize_t write( int fd, const void *buf, size_t nbytes ) {
printf( "XXX: write(%d)\n", fd);
return org_write( fd, buf, nbytes );
}
ssize_t read( int fd, void *buf, size_t nbytes ) {
printf( "XXX: read(%d)\n", fd);
return org_read( fd, buf, nbytes );
}
int close( int fd ) {
printf( "XXX: close()\n");
return org_close( fd );
}
Makefile:
all:
sed 's/XXX/A/g' strcpy.c >strcpyA.c
gcc -fpic -shared -o strcpyA.so strcpyA.c -ldl
sed 's/XXX/B/g' strcpy.c >strcpyB.c
gcc -fpic -shared -o strcpyB.so strcpyB.c -ldl
sed 's/XXX/C/g' strcpy.c >strcpyC.c
gcc -fpic -shared -o strcpyC.so strcpyC.c -ldl
clean:
rm -f *.so strcpyA.c strcpyB.c strcpyC.c
Test:
LD_PRELOAD=/root/strcpy/strcpyA.so:/root/strcpy/strcpyB.so:/root/strcpy/strcpyC.so cat Makefile
Bello: Uh, you are using C++. Thats weired.
Joghurt: Like I said, I'm not that good in C/C++, I'll still have to try it on my TT. :) But it works on my local machine. Why weired?
Bello: Oh, its confusing to me. I only use C.
Wrapper sample
A sample can be downloaded from http://www.homeunix.de/tt/ttwrapper.tgz.
Just edit the wrapper.c file and compile them by
make CC=<path-to-arm-gcc>
Solution Candidate
Escor did a solution in C that can be found here: https://sourceforge.net/projects/ttninit/ Also see http://ttninit.svn.sourceforge.net/viewvc/ttninit/ttninit/. Feel free to join the project!
The C solution is faster than a bash script because the file system timings to load a program for a bash script like "[", "cat", "grep"... are very slow. Also we developed a template for the wrapper to avoid segmentation faults by using more than 1 wrapper for the ttn process.
Joghurt: We need the possibility to define the order in which to chain wrappers, so "data modifying wrappers" are chained directly in front of "ttn" while "data logging wrappers" are chained directly behind the chipset. You can imagine what happens if these things get mixed up when there are more out there, and there will be... How about giving them names with Numbers at the beginning, like "50_testwrapper.so" or so and sorting them when chaining? That would be easy to implement and maintain and would not need anyone to modify a configfile or things like that when installing.
Another proposal, from Vincent Roussillat, Tripmaster's author
Facing v8.350 navcore delivery, that made my plugins merely unusable, I've studied a solution, based on your idea. I came to the following I want to submit to your comments :
- I developed a hook program, launched thru the ttn script. This hook program traps open on /dev/fb, mmap on the framebuffer and memcpy on both framebuffer pages.
- This hook program has got a configuration file telling him which plugins may use its services : one line per plugin giving the name of a file that will trigger the hook action
- To use its services, a plugin only needs to create specific file, named as defined in the configuration, when he wants to take control of the framebuffer. This allow a plugin to run in background (without a trigger file) or in foreground, with a trigger file.
The only thing remaining to make this solution universal is to defined a way to tell the hook program to perform some other actions than the one I have defined that suit my needs. I'm ready to discuss this with all of you.
Joghurt: Good idea, and thank you for providing that tool! Hmm, I would suggest to not use a standalone ttn-file but the ttninit-package, so all our tools could work together and not block each other (I started using it with my new Height build from today), and I would prefer to not use files mentioned in a config-file but a plain single file or something like that, otherwise out tools would need to modify your files on installation time, and could maybe mess things up while doing so. What do you think?
