From ddf5ef9e3f3f891fd6911cd97c6286a6436429b0 Mon Sep 17 00:00:00 2001 From: Michael Grzeschik Date: Tue, 2 Jun 2015 16:23:14 +0200 Subject: docs: rst rework via pandoc Signed-off-by: Michael Grzeschik --- doc/daily_work.rst | 1476 ++++++++++++++++++++++++++++++++++++++++++++++++++ doc/dev_manual.rst | 1324 ++++++++++++++++++++++++++++++++++++++++++++ doc/environment.rst | 687 +++++++++++++++++++++++ doc/getting_help.rst | 84 +++ doc/index.rst | 10 +- doc/ref_manual.rst | 1076 ++++++++++++++++++++++++++++++++++++ doc/revisions.rst | 57 ++ doc/thanks.rst | 14 + doc/user_manual.rst | 883 ++++++++++++++++++++++++++++++ doc/welcome.rst | 133 +++++ 10 files changed, 5743 insertions(+), 1 deletion(-) create mode 100644 doc/daily_work.rst create mode 100644 doc/dev_manual.rst create mode 100644 doc/environment.rst create mode 100644 doc/getting_help.rst create mode 100644 doc/ref_manual.rst create mode 100644 doc/revisions.rst create mode 100644 doc/thanks.rst create mode 100644 doc/user_manual.rst create mode 100644 doc/welcome.rst diff --git a/doc/daily_work.rst b/doc/daily_work.rst new file mode 100644 index 000000000..7982e0e59 --- /dev/null +++ b/doc/daily_work.rst @@ -0,0 +1,1476 @@ +Various Aspects of Daily Work +============================= + +Using an External Kernel Source Tree +------------------------------------ + +This application note describes how to use an external kernel source +tree within a PTXdist project. In this case the external kernel source +tree is managed by GIT. + +Cloning the Linux Kernel Source Tree +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In this example we are using the officially Linux kernel development +tree. + +:: + + jbe@octopus:~$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + [...] + jbe@octopus:~$ ls -l + [...] + drwxr-xr-x 38 jbe ptx 4096 2011-11-16 10:21 myprj + drwxr-xr-x 25 jbe ptx 4096 2011-11-16 10:42 linux + [...] + +Configuring the PTXdist Project +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Note: assumption is here, the directory ``/myprj`` contains a valid +PTXdist project. + +To make PTXdist use of this kernel source tree, instead of an archive we +can simply create a link now: + +:: + + jbe@octopus:~$ cd myprj + jbe@octopus:~/myprj$ mkdir local_src + jbe@octopus:~/myprj$ ln -s ~/linux local_src/kernel.|\ptxdistPlatform{}| + +PTXdist will handle it in the same way as a kernel part of the project. +Due to this, we must setup: + +- Some kind of kernel version + +- Kernel configuration + +- Image type used on our target architecture + +- If we want to build modules + +- Patches to be used (or not) + +Lets setup these topics. We just add the kernel component to it. + +:: + + jbe@octopus:~/myprj$ ptxdist platformconfig + +We must enable the **Linux kernel** entry first, to enable kernel +building as part of the project. After enabling this entry, we must +enter it, and: + +- Setting up the **kernel version** + +- Setting up the **MD5 sum** of the corresponding archive + +- Selecting the correct image type in the entry **Image Type**. + +- Configuring the kernel within the menu entry **patching & + configuration**. + + - If no patches should be used on top of the selected kernel source + tree, we keep the **patch series file** entry empty. As GIT should + help us to create these patches for deployment, it should be kept + empty on default in this first step. + + - Select a name for the kernel configuration file and enter it into + the **kernel config file** entry. + +Even if we do not intend to use a kernel archive, we must setup these +entries with valid content, else PTXdist will fail. Also the archive +must be present on the host, else PTXdist will start a download. + +Now we can leave the menu and store the new setup. The only still +missing component is a valid kernel config file now. We can use one of +the default config files the Linux kernel supports as a starting point. +To do so, we copy one to the location, where PTXdist expects it in the +current project. In a multi platform project this location is the +platform directory usally in ``configs/``. We must +store the file with a name selected in the platform setup menu (**kernel +config file**). + +Work Flow +~~~~~~~~~ + +Now its up to ourself working on the GIT based kernel source tree and +using PTXdist to include the kernel into the root filesystem. + +To configure the kernel source tree, we simply run: + +:: + + jbe@octopus:~/myprj$ ptxdist kernelconfig + +To build the kernel: + +:: + + jbe@octopus:~/myprj$ ptxdist targetinstall kernel + +To rebuild the kernel: + +:: + + jbe@octopus:~/myprj$ ptxdist drop kernel compile + jbe@octopus:~/myprj$ ptxdist targetinstall kernel + +Discovering Runtime Dependencies +-------------------------------- + +Often it happens that an application on the target fails to run, because +one of its dependencies is not fulfilled. This section should give some +hints on how to discover these dependencies. + +Dependencies on Shared Libraries +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Getting the missed shared library for example at runtime is something +easily done: The dynamic linker prints the missing library to the +console. + +To check at build time if all other dependencies are present is easy, +too. The architecture specific ``readelf`` tool can help us here. It +comes with the OSELAS.Toolchain and is called via ``-readelf`` (this +example uses the one coming with the ** toolchain). + +To test the ``foo`` binary from our new package ``FOO``, we simply run: + +:: + + @\$ \ptxdistCompilerName -readelf -d \ptxdistPlatformDir @root/usr/bin/foo | grep NEEDED + 0x00000001 (NEEDED) Shared library: [libm.so.6] + 0x00000001 (NEEDED) Shared library: [libz.so.1] + 0x00000001 (NEEDED) Shared library: [libc.so.6] + +We now can check if all of the listed libraries are present in the root +filesystem. This works for shared libraries, too. It is also a way to +check if various configurations of our package are working as expected +(e.g. disabling a feature should also remove the required dependency of +this feature). + +Dependencies on other Resources +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Sometimes a binary fails to run due to missing files, directories or +device nodes. Often the error message (if any) that the binary creates +in this case is ambiguous. Here the ``strace`` tool can help us, namely +to observe the binary at runtime. ``strace`` shows all the system calls +that the binary or its shared libraries are performing. + +``strace`` is one of the target debugging tools which PTXdist provides +in its ``Debug Tools`` menu. + +After adding strace to the root filesystem, we can use it and observe +our ``foo`` binary: + +:: + + @\$@ strace usr/bin/foo + execve("/usr/bin/foo", ["/usr/bin/foo"], [/* 41 vars */]) = 0 + brk(0) = 0x8e4b000 + access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) + open("/etc/ld.so.cache", O_RDONLY) = 3 + fstat64(3, {st_mode=S_IFREG|0644, st_size=77488, ...}) = 0 + mmap2(NULL, 77488, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f87000 + close(3) = 0 + open("/lib//lib/libm-2.5.1.so", O_RDONLY) = 3 + read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p%\0\000"..., 512) = 512 + mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f86000 + fstat64(3, {st_mode=S_IFREG|0555, st_size=48272, ...}) = 0 + mmap2(NULL, 124824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7f67000 + mmap2(0xb7f72000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xb) = 0xb7f72000 + mmap2(0xb7f73000, 75672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f73000 + close(3) = 0 + open("/lib/libc.so.6", O_RDONLY) = 3 + read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\332X\1"..., 512) = 512 + fstat64(3, {st_mode=S_IFREG|0755, st_size=1405859, ...}) = 0 + [...] + +Occasionally the output of ``strace`` can be very long and the +interesting parts are lost. So, if we assume the binary tries to open a +nonexisting file, we can limit the output to all ``open`` system calls: + +:: + + @\$@ strace -e open usr/bin/foo + open("/etc/ld.so.cache", O_RDONLY) = 3 + open("/lib/libm-2.5.1.so", O_RDONLY) = 3 + open("/lib/libz.so.1.2.3", O_RDONLY) = 3 + open("/lib/libc.so.6", O_RDONLY) = 3 + [...] + open("/etc/foo.conf", O_RDONLY) = -1 ENOENT (No such file or directory) + +The binary may fail due to a missing ``/etc/foo.conf``. This could be a +hint on what is going wrong (it might not be the final solution). + +Debugging with CPU emulation +---------------------------- + +|image| + +If we do not need some target related feature to run our application, we +can also debug it through a simple CPU emulation. Thanks to QEMU we can +run ELF binaries for other architectures than our build host is. + +Running an Application made for a different Architecture +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PTXdist creates a fully working root filesystem with all runtime +components in ``root/``. Lets assume we made a PTXdist based project for +a CPU. Part of this project is our application ``myapp`` we are +currently working on. PTXdist builds the root filesystem and also +compiles our application. It also installs it to ``usr/bin/myapp`` in +the root filesystem. + +With this preparation we can run it on our build host: + +:: + + $ cd |\ptxdistPlatformDir|root + |\ptxdistPlatformDir|root |\$ \ptxdistQEMUName{}| -cpu |\ptxdistQEMUCPU{}| -L . usr/bin/myapp + +This command will run the application ``usr/bin/myapp`` built for an CPU +on the build host and is using all library compontents from the current +directory. + +For the stdin and -out QEMU uses the regular mechanism of the build +host’s operating system. Using QEMU in this way let us simply check our +programs. There are also QEMU environments for other architectures +available. + +Debugging an Application made for a different Architecture +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Debugging our application is also possible with QEMU. All we need are a +root filesystem with debug symbols available, QEMU and an architecture +aware debugger. + +The root filesystem with debug symbols will be provided by PTXdist, the +architecture aware debugger comes with the OSELAS.Toolchain. Two +consoles are required for this debug session in this example. We start +the QEMU in the first console as: + +:: + + $ cd |\ptxdistPlatformDir|root-debug + |\ptxdistPlatformDir|root-debug |\$ \ptxdistQEMUName{}| -g 1234 -cpu |\ptxdistQEMUCPU{}| -L . usr/bin/myapp + +Note: PTXdist always builds two root filesystems. ``root/`` and +``root-debug/``. ``root/`` contains all components without debug +information (all binaries are in the same size as used later on on the +real target), while all components in ``root-debug/`` still containing +the debug symbols and are much bigger in size. + +The added *-g 1234* parameter lets QEMU wait for a GDB connection to run +the application. + +In the second console we start GDB with the correct architecture +support. This GDB comes with the same OSELAS.Toolchain that was also +used to build the project. In our example we are using the ** toolchain +for our CPU: + +:: + + |\$ \ptxdistCompilerName|-gdbtui root-debug/usr/bin/myapp + +This will run a *curses* based GDB. Not so easy to handle (we must enter +all the commands and cannot click with a mouse!), but very fast to take +a quick look at our application. + +At first we tell GDB where to look for debug symbols. The correct +directory here is ``root-debug/``. + +:: + + (gdb) set solib-absolute-prefix |\ptxdistPlatformDir|root-debug + +Next we connect this GDB to the waiting QEMU: + +:: + + (gdb) target remote localhost:1234 + Remote debugging using localhost:1234 + [New Thread 1] + 0x40096a7c in _start () from root-debug/lib/ld.so.1 + +As our application is already started, we can’t use the GDB command +``start`` to run it until it reaches ``main()``. We set a breakpoint +instead at ``main()`` and *continue* the application: + +:: + + (gdb) break main + Breakpoint 1 at 0x100024e8: file myapp.c, line 644. + (gdb) continue + Continuing. + Breakpoint 1, main (argc=1, argv=0x4007f03c) at myapp.c:644 + +The top part of the running gdbtui console will always show us the +current source line. Due to the ``root-debug/`` directory usage all +debug information for GDB is available. + +Now we can step through our application by using the commands *step*, +*next*, *stepi*, *nexti*, *until* and so on. + +Note: It might be impossible for GDB to find debug symbols for +components like the main C runtime library. In this case they where +stripped while building the toolchain. There is a switch in the +OSELAS.Toolchain menu to keep the debug symbols also for the C runtime +library. But be warned: This will enlarge the OSELAS.Toolchain +installation on your harddisk! When the toolchain was built with the +debug symbols kept, it will be also possible for GDB to debug C library +functions our application calls (so it might worth the disk space). + +Migration between Releases +-------------------------- + +To migrate an existing project from within one minor release to the next +one, we do the following step: + +:: + + ~/my_bsp# ptxdist migrate + +PTXdist will ask us for every new configuration entry what to do. We +must read and answer these questions carefully. At least we shouldn’t +answer blindly with ’Y’ all the time because this could lead into a +broken configuration. On the other hand, using ’N’ all to time is more +safer. We can still enable interesting new features later on. + +Software Installation and Upgrade +--------------------------------- + +Root filesystems for Linux are usually built as a flash image and pushed +into the respective root medium. However, when working with Embedded +Linux systems, developers often have the need to + +- install new packages + +- remove packages + +- update packages + +- add configuration + +Installation of new packages may for example happen if a developer works +on a new piece of application code, or if a new library is being written +for the embedded system. Package updating may be a requirement even +during the system’s life cycle, for example for updating a customer +application in the field. + +Conventional Linux distributions like Debian, SuSE or Fedora use package +systems like RPM or DEB to organize their software packages. +Unfortunately, these methods require huge packet databases in the root +system, which is bad for space constrained embedded systems. So what we +need is a packet system that + +- offers installation/removement of packages + +- has no big database but a very low overhead + +- allows packet management features like pre/post scripts (i.e. + shutdown a web server, then upgrade it and start it again) + +``ipkg`` is such a packet system and it is being used in ptxdist. +Originally developed for the IBM Itsy, ``ipkg`` is meanwhile being used +on all kinds of embedded Linux projects. The concept of ``ipkg`` +archives is based on the well known Debian packet management format: +``ipkg`` archives are “ar” archives, containing a tarball with the +binary files for the target box, plus management scripts which can be +run on pre-install, post-install, pre-rm and post-rm. So even if no +``ipkg`` management utilities are available, developers can modify the +archives with standard shell and console tools. + +ipkg Usage in PTXdist +~~~~~~~~~~~~~~~~~~~~~ + +PTXdist end users and packet developers don’t have to care directly +about ``ipkg``. Packages are being created during the *targetinstall* +stage, then put into the ``packages/`` directory. After the +targetinstall stage of a packet was done, this directory contains the +``ipkg`` packet itself plus, for most packages, a directory with the +file content. + +The ``ipkg`` packets contain the binaries for the root filesystem as +well as start/stop scripts and meta data about the Unix filesystem +permissions; when PTXdist creates the root filesystem which is later +flashed into the onboard flash of an embedded system, it takes the +information from the ``ipkg`` packets as the source, in order to make +sure that the image is consistent to what the packages contain. + +Internally, PTXdist always uses ``ipkg`` packets to store it’s target +data. However, the ``ipkg`` functionality is not always exposed to the +target itself. So in order to use packets, navigate to *Disk and File +Utilities* and enable ``ipkg``. In the ``ipkg`` sub menu, make sure that +the *install /etc/ipkg.conf* switch is active. This config file is +necessary to make ``ipkg`` work at runtime system. + +The *ipkg* tool can only be used in images created by +``ptxdist images``. It’s not fully working within the ``root/`` +subdirectory used as NFS root filesystem. + +Packet Installation +~~~~~~~~~~~~~~~~~~~ + +A common use case for ``ipkg`` packets is to push new software to an +already deployed target. There must be a communication channel to +transfer the packet itself to the embedded system, i.e. by using FTP or +a secure connection via SFTP or SSH, so it has to be made sure that such +a service is installed and configured on the target. It is necessary +that enough free space is available, in order to store the packet +itself. A good rule of thumb is to have twice the size of the installed +package: while the packet is being installed, the archive as well as +it’s contents must fit into the system storage. This may be a problem on +space constrained systems. + +If the packet was transferred, it is necessary to have remote shell +access to the box, either via telnet or, if security is an issue, by +using SSH. It is also possible to automate this process by using an +intelligent update mechanism. The shell is being used to start the +necessary commands. If the packet was installed, the ``ipkg`` archive +can be removed again. + +Automatic Packet Download +~~~~~~~~~~~~~~~~~~~~~~~~~ + +It is also possible to let the embedded system download ``ipkg`` packets +automatically from a network source, without pushing the packets from +the outside. In order to do so, a valid URL must be written into the +``/etc/ipkg.conf`` file. In this case one of the ``wget`` +implementations in PTXdist must be selected, either the one in busybox +(**Shell & Console Tools**, **BusyBox**, **Networking Utilities**) or +the native implementation (**Networking Tools**). + +The ipkg Command +~~~~~~~~~~~~~~~~ + +The following sections describe the ``ipkg`` features. + +What’s Installed on the System? +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To get a list of installed packages, use *list\_installed*: + +:: + + # ipkg list_installed + busybox - 1.1.3 - + figlet - 222 - + gcclibs - 4.1.1 - + gdbserver - 6.4 - + glib - 2.8.6 - + glibc - 2.5 - + ipkg - 0.99.163 - + ixp-firmware - 1 - + kernel-modules - 2.6.18 - + libxml2 - 2.6.27 - + mc - 4.6.1 - + memedit - 0.7 - + ncurses - 5.5 - + pciutils - 2.2.1 - + pureftpd - 1.0.21 - + readline - 5.0 - + rootfs - 1.0.0 - + strace - 4.5.14-20061101 - + udev - 088 - + zlib - 1.2.3 - + Successfully terminated. + +Content of a Package +^^^^^^^^^^^^^^^^^^^^ + +To see what files are in an installed package, use *files*: + +:: + + # ipkg files udev + Package udev (106) is installed on root and has the following files: + /etc/init.d/udev + /sbin/udevtrigger + /etc/udev/udev.conf + /etc/rc.d/S00_udev + /sbin/udevd + /sbin/udevsettle + + Successfully terminated. + +Adding a Package +^^^^^^^^^^^^^^^^ + +Adding a new packet or replacing an already installed one is done by + +:: + + # ipkg install .ipk + +Note the trailing **.ipk**. This extension must be given if the package +file is already part of the filesystem. Otherwise ``ipkg`` tries to +download it from the URL configured in ``/etc/ipkg.conf``. + +Removing a Package +^^^^^^^^^^^^^^^^^^ + +To remove the contents of a package from the running system, ensure that +nothing from the package is currently in use. Find out the precise +packet name with + +:: + + # ipkg list + +and remove it’s contents from the runtime system with + +:: + + # ipkg remove + +Upgrading a Package +^^^^^^^^^^^^^^^^^^^ + +To upgrade a package, first remove it’s current contents from the +runtime system. In a second step, install the contents of the new +``ipkg`` package. + +:: + + # ipkg list + # ipkg remove + # ipkg install [.ipk] + +Increasing Build Speed +---------------------- + +Modern host systems are providing more than one CPU core. To make use of +this additionally computing power recent applications should do their +work in parallel. + +|image| + +Using available CPU Cores +~~~~~~~~~~~~~~~~~~~~~~~~~ + +PTXdist uses all available CPU cores when building a project by default. +But there are some exceptions: + +- the prepare stage of all autotools build system based packages can + use only one CPU core. This is due to the fact, the running + “configure” is a shell script. + +- some packages have a broken buildsystem regarding parallel build. + These kind of packages build successfully only when building on one + single CPU core. + +- creating the root filesystem images are also done on a single core + only + +Manually adjusting CPU Core usage +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Manual adjustment of the parallel build behaviour is possible via +command line parameters. + +-ji + this defines the number of CPU cores to build a package. The default + is two times the available CPU cores. + +-je + this defines the number of packages to be build in parallel. The + default is one package at a time. + +-l + limit the system load to the given value. + +Building packages in parallel is still not tested very well and migh +fail. + +Building in Background +~~~~~~~~~~~~~~~~~~~~~~ + +To build a project in background PTXdist can be ’niced’. + +-n[ + ] run PTXdist and all of its child processes with the given + nicelevel . Without a nicelevel the default is 10. + +Building Platforms in Parallel +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Due to the fact that more than one platform can exist in a PTXdist +project, all these platforms can be built in parallel within the same +project directory. As they store their results into different platform +subdirectories, they will not conflict. Only PTXdist must be called +differently, because each call must be parametrized individually. + +The used Platform Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + $ ptxdist platform + +This call will create the soft link ``selected_platformconfig`` to the +```` in the project’s directory. After this call, +PTXdist uses this soft link as the default platform to build for. + +It can be overwritten temporarily by the command line parameter +``--platformconfig=``. + +The used Project Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + $ ptxdist select + +This call will create the soft link ``selected_ptxconfig`` to the +```` in the project’s directory. After this call, +PTXdist uses this soft link as the default configuration to build the +project. + +It can be overwritten temporarily by the command line parameter +``--ptxconfig=``. + +The used Toolchain to Build +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + $ ptxdist toolchain + +This call will create the soft link ``selected_toolchain`` to the +```` in the project’s directory. After this call, +PTXdist uses this soft link as the default toolchain to build the +project with. + +It can be overwritten temporarily by the command line parameter +``--toolchain=``. + +By creating the soft links all further PTXdist commands will use these +as the default settings. + +By using the three ``--platformconfig``, ``--ptxconfig`` and +``--toolchain`` parameters, we can switch (temporarily) to a completely +different setting. This feature we can use to build everything in one +project. + +A few Examples +^^^^^^^^^^^^^^ + +The project contains two individual platforms, sharing the same +architecture and same project configuration. + +:: + + $ ptxdist select + $ ptxdist toolchain + $ ptxdist --platformconfig= --quiet go & + $ ptxdist --platformconfig= go + +The project contains two individual platforms, sharing the same project +configuration. + +:: + + $ ptxdist select + $ ptxdist --platformconfig= --toolchain= --quiet go & + $ ptxdist --platformconfig= --toolchain= go + +The project contains two individual platforms, but they do not share +anything else. + +:: + + $ ptxdist --select= --platformconfig= --toolchain= --quiet go & + $ ptxdist --select= --platformconfig= --toolchain= go + +Running one PTXdist in background and one in foreground would render the +console output unreadable. That is why the background PTXdist uses the +``--quiet`` parameter in the examples above. Its output is still +available in the logfile under the platform build directory tree. + +By using more than one virtual console, both PTXdists can run with their +full output on the console. + +Using a Distributed Compiler +---------------------------- + +To increase the build speed of a PTXdist project can be done by doing +more tasks in parallel. PTXdist itself uses all available CPU cores by +default, but is is limited to the local host. For further speedup a +distributed compilation can be used. This is the task of *ICECC* aka +*icecream*. With this feature a PTXdist project can make use of all +available hosts and their CPUs in a local network. + +Setting-Up the Distributed Compiler +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +How to setup the distributed compiler can be found on the project’s +homepage at GITHUB: + +https://github.com/icecc/icecream. + +Read their ``README.md`` for further details. + +Please note: as of July 2014 you need at least an *ICECC* in its version +1.x. Older revisions are known to not work. + +Enabling PTXdist for the Distributed Compiler +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Since the 2014.07 release, PTXdist supports the usage of *ICECC* by +simply enabling a setup switch. + +Run the PTXdist setup and navigate to the new *ICECC* menu entry: + +:: + + $ ptxdist setup + Developer Options ---> + [*] use icecc + (/usr/lib/icecc/icecc-create-env) icecc-create-env path + +Maybe you must adapt the ``icecc-create-env path`` to the setting on +your host. Most of the time the default path should work. + +How to use the Distributed Compiler with PTXdist +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PTXdist still uses two times the count of cores of the local CPU for +parallel tasks. But if a faster CPU in the net exists, *ICECC* will now +start to do all compile tasks on this/these faster CPU(s) instead of the +local CPU. + +To really boost the build speed you must increase the tasks to be done +in parallel manually. Use the ``-ji`` command line option to start +more tasks at the same time. This command line option just effects one +package to build at a time. To more increase the build speed use the +``-je`` command line option as well. This will build also packages in +parallel. + +A complete command line could look like this: + +:: + + $ ptxdist go -ji64 -je8 + +This command line will run up to 64 tasks in parallel and builds 8 +packages at the same time. Never worry again about your local host and +how slow it is. With the help of *ICECC* every host will be a high speed +development machine. + +Using pre-build archives +------------------------ + +PTXdist is a tool which creates all the required parts of a target’s +filesystem to breathe life into it. And it creates these parts from any +kind of source files. If a PTXdist project consists of many packages the +build may take a huge amount of time. + +For internal checks we have a so called “ALL-YES” PTXdist project. It +has - like the name suggests - all packages enabled which PTXdist +supports. To build this “ALL-YES” PTXdist project our build server needs +about 6 hours. + +Introduction +~~~~~~~~~~~~ + +While the development of a PTXdist project it is needed to clean and +re-build everything from time to time to get a re-synced project result +which honors all changes made in the project. But since cleaning and +re-building everything from time to time is a very good test case if +some adaptions are still missing or if everything is complete, it can be +a real time sink to do so. + +To not lose developer’s temper when doing such tests, PTXdist can keep +archives from the last run which includes all the files the package’s +build system has installed while the PTXdist’s *install* stage runs for +it. + +The next time PTXdist should build a package it can use the results from +the last run instead. This feature can drastically reduce the time to +re-build the whole project. But also, this PTXdist feature must handle +with care and so it is not enabled and used as default. + +This section describes how to make use of this PTXdist feature and what +pitfalls exists when doing so. + +Creating pre-build archives +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To make PTXdist creating pre-build archives, enable this feature prior a +build in the menu: + +:: + + $ ptxdist menuconfig + + Project Name & Version ---> + [*] create pre-build archives + +Now run a regular build of the whole project: + +:: + + $ ptxdist go + +When the build is finished, the directory ``packages`` contains +additional archives files with the name scheme ``*-dev.tar.gz``. These +files are the pre-build archives which PTXdist can use later on to +re-build the project. + +Using pre-build archives +~~~~~~~~~~~~~~~~~~~~~~~~ + +To make PTXdist using pre-build archives, enable this feature prior a +build in the menu: + +:: + + $ ptxdist menuconfig + + Project Name & Version ---> + [*] use pre-build archives + () + +With the next build (e.g. ``ptxdist go``) PTXdist will look for a +specific package if its corresponding pre-build archive does exist. If +it does exist and the used hash value in the pre-build archive’s +filename matches, PTXdist will skip all source archive handling +(extract, patch, compile and install) and just extract and use the +pre-build archive’s content. + +A regular and save usecase of pre-build archives is: + +- using one pre-build archive pool for one specific PTXdist project. + +- using a constant PTXdist version all the time. + +- using a constant OSELAS.Toolchain() version all the time. + +- no package with a pre-build archive in the project is under + development. + +The hash as a part of the pre-build archive’s filename does only reflect +the package’s configuration made in the menu (``ptxdist menuconfig``). +If this package specific configuration changes, a new hash value will be +the result and PTXdist can select the corresponding correct pre-build +archive. + +This hash value change is an important fact, as many things outside and +inside the package can have a big impact of the binary result but +without a hash value change! + +Please be careful when using the pre-build archives if you: + +- intend to switch to a different toolchain with the next build. + +- change the patch set applied to the corresponding package, e.g. a + package is under development. + +- change the hard coded configure settings in the package’s rule file, + e.g. a package is under development + +- intend to use one pre-build archive pool from different PTXdist + projects. + +To consider all these precautions the generated pre-build archives are +not transfered automatically where the next build expects them. This +must be done manually by the user of the PTXdist project. Doing so, we +can decide on a package by package base if its pre-build archive should +be used or not. + +Packages without pre-build archives support +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +No host nor cross related packages can be used based on their pre-build +archives. These kind of packages are always (re-)built. + +Only target related packages can be used based on their pre-build +archives, but there are a few exceptions: + +- Linux kernel: It has an incomplete install stage, which results into + an incomplete pre-build archive. Due to this, it cannot be used as a + pre-build archive + +- Barebox bootloader: It has an incomplete install stage, which results + into an incomplete pre-build archive. Due to this, it cannot be used + as a pre-build archive + +- some other somehow broken packages all marked with a + ``_DEVPKG := NO`` in their corresponding rule file + +Workflow with pre-build archives +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We starting with an empty PTXdist project and enabling the pre-build +archive feature as mentioned in [enabling:sub:`p`\ rebuild]. After that +a regular build of the project can be made. + +When the build is finished its time to copy all the pre-build archives +of interest where the next build will expect them. +[using:sub:`p`\ rebuild] mentions the step to enable their use. It also +allows to define a directory. The default path of this directory is made +from various other menu settings, to ensure the pre-build archives of +the current PTXdist project do not conflict with pre-build archives of +different PTXdist projects. To get an idea what the final path is, we +can ask PTXdist. + +:: + + $ ptxdist print PTXCONF_PROJECT_DEVPKGDIR + /home/jbe/OSELAS.BSP/|\ptxdistHwVendor /\ptxdistBSPName{}| + +If this directory does not exist, we can simply create it: + +:: + + $ mkdir -p /home/jbe/OSELAS.BSP/|\ptxdistHwVendor /\ptxdistBSPName{}| + +Now its time to copy the pre-build archives to this new directory. We +could simply copy all pre-build archives from the ``/packages`` +directory. But we should keep in mind, if any of the related packages +are under development, we must omit its corresponding pre-build archives +in this step. + +:: + + $ cp |\ptxdistPlatformDir packages/*-dev.tar.gz| /home/jbe/OSELAS.BSP/|\ptxdistHwVendor /\ptxdistBSPName{}| + +Uses cases +~~~~~~~~~~ + +Some major possible use cases are covered in this section: + +- Speed up a re-build of one single project + +- Sharing pre-build archives between two platforms based on the same + architecture + +To simply speed up a re-build of the whole project (without development +on any of the used packages) we just can copy all ``*-dev.tar.gz`` +archives after the first build to the location where PTXdist expects +them at the next build time. + +If two platforms are sharing the same architecture it is possible to +share pre-build archives as well. The best way it can work is, if both +platforms are part of the same PTXdist project. They must also share the +same toolchain settings, patch series and rule files. If these +precautions are handled the whole project can be built for the first +platform and these pre-build archives can be used to build the project +for the second platform. This can reduce the required time to build the +second platform from hours to minutes. + +Downloading Packages from the Web +--------------------------------- + +Sometimes it makes sense to get all required source archives at once. +For example prior to a shipment we want to also include all source +archives, to free the user from downloading it by him/herself. + +PTXdist supports this requirement by the ``export_src`` parameter. It +collects all required source archives into one given single directory. +To do so, the current project must be set up correctly, e.g. the +``select`` and ``platform`` commands must be ran prior the +``export_src`` step. + +If everything is set up correctly we can run the following commands to +get the full list of required archives to build the project again +without an internet connection. + +:: + + $ mkdir my_archives + $ ptxdist export_src my_archives + +PTXdist will now collect all source archives to the ``my_archives/`` +directory. + +Note: If PTXdist is configured to share one source archive directory for +all projects, this step will simply copy the source archives from the +shared source archive directory. Otherwise PTXdist will start to +download them from the world wide web. + +Creating a new Autotools Based Library +-------------------------------------- + +Developing your own library can be one of the required tasks to support +an embedded system. PTXdist comes with an autotoolized library template +which can be simply integrated into a PTXdist project. + +Creating the Library Template +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Creating the library package can be done by the PTXdist’s *newpackage* +command: + +:: + + > ptxdist newpackage src-autoconf-lib + + ptxdist: creating a new 'src-autoconf-lib' package: + + ptxdist: enter package name...........: foo + ptxdist: enter version number.........: 1 + ptxdist: enter package author.........: Juergen Borleis + ptxdist: enter package section........: project_specific + + generating rules/foo.make + generating rules/foo.in + + local_src/foo does not exist, create? [Y/n] Y + ./ + ./internal.h + ./@name@.c + ./configure.ac + ./README + ./COPYING + ./Makefile.am + ./lib@name@.pc.in + ./autogen.sh + ./lib@name@.h + ./wizard.sh + +After this step the new directory ``local_src/foo`` exists and contains +various template files. All of these files are dedicated to be modified +by yourself. + +The content of this directory is: + +:: + + > ls -l local_src/foo/ + total 48 + -rw-r--r-- 1 jbe ptx 335 Jun 18 23:00 COPYING + -rw-r--r-- 1 jbe ptx 1768 Jun 18 23:16 Makefile.am + -rw-r--r-- 1 jbe ptx 1370 Jun 18 23:16 README + -rwxr-xr-x 1 jbe ptx 267 Apr 16 2012 autogen.sh + -rw-r--r-- 1 jbe ptx 11947 Jun 18 23:16 configure.ac + -rw-r--r-- 1 jbe ptx 708 Jun 18 23:16 foo.c + -rw-r--r-- 1 jbe ptx 428 Jun 18 23:00 internal.h + -rw-r--r-- 1 jbe ptx 185 Jun 18 23:16 libfoo.h + -rw-r--r-- 1 jbe ptx 331 Jun 18 23:16 libfoo.pc.in + drwxr-xr-x 2 jbe ptx 4096 Jun 18 23:16 m4 + +Licence related stuff +~~~~~~~~~~~~~~~~~~~~~ + +COPYING +^^^^^^^ + +You must think about the licence your library uses. The template file +``COPYING`` contains some links to GPL/LGPL texts you can use. Replace +the ``COPYING``\ ’s content by one of the listed licence files or +something different. But do not omit this step. Never! + +Build system related files +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +autogen.sh +^^^^^^^^^^ + +The autotools are using macro files which are easier to read for a +human. But to work with the autotools these macro files must be +converted into executabe shell code first. The ``autogen.sh`` script +does this job for us. + +configure.ac +^^^^^^^^^^^^ + +This is the first part of the autotools based build system. Its purpose +is to collect as much required information as possible about the target +to build the library for. This file is a macro file. It uses a bunch of +M4 macros to define the job to do. The autotools are complex and this +macro file should help you to create a useful and cross compile aware +``configure`` script everybody can use. + +This macro file is full of examples and comments. Many M4 macros are +commented out and you can decide if you need them to detect special +features about the target. + +Search for the “TODO” keyword and adapt the setting to your needs. After +that you should remove the “TODO” comments to not confuse any user later +on. + +Special hints about some M4 macros: + +AC\_INIT + add the intended revision number (the second argument), an email + address to report bugs and some web info about your library. The + intended revision number will be part of the released archive name + later on. You can keep it in sync with the API\_RELEASE, but you + must not. + +AC\_PREFIX\_DEFAULT + most of the time you can remove this entry, because most users + expect the default install path prefix is ``/usr/local`` which is + always the default if not changed by this macro. + +API\_RELEASE + defines the API version of your library. This API version will be + part of the binary library’s name later on. + +LT\_CURRENT / LT\_REVISION / LT\_AGE + define the binary compatibility of your library. The rules how these + numbers are defined are: + + - library code was modified: ``LT_REVISION++`` + + - interfaces changed/added/removed: ``LT_CURRENT++`` and + ``LT_REVISION = 0`` + + - interfaces added: ``LT_AGE++`` + + - interfaces removed: ``LT_AGE = 0`` + + You must manually change these numbers whenever you change the code + in your library prior a release. + +CC\_CHECK\_CFLAGS / CC\_CHECK\_LDFLAGS + if you need special command line parameters given to the compiler or + linker, don’t add them unconditionally. Always test, if the tools + can handle the parameter and fail gracefully if not. Use + CC\_CHECK\_CFLAGS to check parameters for the compiler and + CC\_CHECK\_LDFLAGS for the linker. + +AX\_HARDWARE\_FP / AX\_DETECT\_ARMV\* + sometimes it is important to know for which architecture or CPU the + current build is for and if it supports hard float or not. Please + don’t try to guess it. Ask the compiler instead. The M4 + AX\_HARDWARE\_FP and AX\_DETECT\_ARMV\* macros will help you. + +REQUIRES + to enrich the generated \*.pc file for easier dependency handling + you should also fill the REQUIRES variable. Here you can define from + the package management point of view the dependencies of your + library. For example if your library depends on the ’udev’ library + and requires a specific version of it, just add the string + ``udev >= 1.0.0`` to the REQUIRES variable. Note: the listed + packages must be comma-separated. + +CONFLICTS + if your library conflicts with a different library, add this + different library to the CONFLICTS variable (from the package + management point of view). + +It might be a good idea to include the API version into the names of the +library’s include file and pkg-config file. For example in the first API +version all files are named like this: + +- /usr/local/lib/libfoo-1.so.0.0.0 + +- /usr/local/include/libfoo-1.h + +- /usr/local/lib/pkgconfig/libfoo-1.pc + +In this case its simple to create the next generation libfoo without +conflicting with earlier versions of your library: they can co-exist +side by side. + +- /usr/local/lib/libfoo-1.so.0.0.0 + +- /usr/local/lib/libfoo-2.so.0.0.0 + +- /usr/local/include/libfoo-1.h + +- /usr/local/include/libfoo-2.h + +- /usr/local/lib/pkgconfig/libfoo-1.pc + +- /usr/local/lib/pkgconfig/libfoo-2.pc + +If you want to do so, you must rename the header file and the pc file +accordingly. And adapt the *pkgconfig\_DATA* and *include\_HEADERS* +entries in the ``Makefile.am`` file, and the *AC\_CONFIG\_FILES* in the +``configure.ac`` file. + +Makefile.am +^^^^^^^^^^^ + +Special hints: + +SUBDIR + if your project contains more than one sub-directory to build, add + these directories here. Keep in mind, these directories are visited + in this order (but never in parallel), so you must handle + dependencies manually. + +\*\_CPPFLAGS / \*\_CFLAGS / \*\_LIBADD + if your library has some optional external dependencies add them on + demand (external libraries for example). Keep in mind to not mix + CPPFLAGS and CFLAGS additions. And do not add these additions fixed + to the \*\_CPPFLAGS and \*\_CFLAGS variables, let ’configure’ do it + in a sane way. Whenever you want to forward special things to the + \*\_CPPFLAGS and \*\_CFLAGS, don’t forget to add the AM\_CPPFLAGS + and AM\_CFLAGS, else they get lost. Never add libraries to the + \*\_LDFLAGS variable. Always add them to the \*\_LIBADD variable + instead. This is important because the autotools forward all these + variable based parameters in a specifc order to the tools (compiler + and linker). + +Template file for pkg-config +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +libfoo.pc.in +^^^^^^^^^^^^ + +This file gets installed to support the *pkg-config* tool for package +management. It contains some important information how to use your +library and also handles its dependencies. + +Special hints: + +Name + A human-readable name for the library. + +Description + add a brief description of your library here + +Version + the main revision of the library. Will automatically replaced from + your settings in ``configure.ac``. + +URL + where to find your library. Will automatically replaced from your + settings in ``configure.ac``. + +Requires.private + comma-separated list of packages your library depends on and managed + by pkg-config. The listed packages gets honored for the static + linking case and should not be given again in the *Libs.private* + line. This line will be filled by the *REQUIRES* variable from the + ``configure.ac``. + +Conflicts + list of packages your library conflicts with. Will automatically + replaced from your CONFLICTS variable settings in ``configure.ac``. + +Libs + defines the linker command line content to link your library against + other applications or libraries + +Libs.private + | defines the linker command line content to link your library + against other application or libraries statically. List only + libraries here which are not managed by pkg-config (e.g. do not + conflict with packages given in the *Requires*). + | This line will be filled by the *LIBS* variable from the + ``configure.ac``. + +Cflags + required compile flags to make use of your library. Unfortunately + you must mix CPPFLAGS and CFLAGS here which is a really bad idea. + +It is not easy to fully automate the adaption of the pc file. At least +the lines *Requires.private* and *Libs.private* are hardly to fill for +packages which are highly configureable. + +Generic template files +~~~~~~~~~~~~~~~~~~~~~~ + +m4/\* +^^^^^ + +M4 macros used in ``configure.ac``. + +If you use more no generic M4 macros in your ``configure.ac`` file, +don’t forget to add their source files to the m4 directory. This will +enable any user of your library to re-generate the autotools based files +without providing all dependencies by themself. + +Library related files +~~~~~~~~~~~~~~~~~~~~~ + +README +^^^^^^ + +Prepared file with some information about the library you provide. Be +kind to the users of your library and write some sentences about basic +features and usage of your library, how to configure it and how to build +it. + +libfoo.h +^^^^^^^^ + +This file will be installed. It defines the API your library provides +and will be used by other applications. + +internal.h +^^^^^^^^^^ + +This file will not be installed. It will be used only at build time of +your library. + +foo.c +^^^^^ + +The main source file of your library. Keep in mind to mark all functions +with the DSO\_VISIBLE macro you want to export . All other functions are +kept internaly and you cannot link against them from an external +application. + +Note: debugging is hard when all internal functions are hidden. For this +case you should configure the libary with the ``--disable-hide`` or with +``--enable-debug`` which includes switching off hiding functions. + +Frequently Asked Questions (FAQ) +-------------------------------- + +| Q: PTXdist does not support to generate some files in a way I need + them. What can I do? +| A: Everything PTXdist builds is controlled by “package rule files”, + which in fact are Makefiles (``rules/*.make``). If you modify such a + file you can change it’s behaviour in a way you need. It is generally + no good idea to modify the generic package rule files installed by + PTXdist, but it is always possible to copy one of them over into the + ``rules/`` directory of a project. Package rule files in the project + will precede global rule files with the same name. + +| Q: My kernel build fails. But I cannot detect the correct position, + due to parallel building. How can I stop PTXdist to build in parallel? +| A: Force PTXdist to stop building in parallel which looks somehow + like: + +:: + + > ptxdist -j1 go + +Q: I made my own rule file and now I get error messages like + +:: + + my_project/rules/test.make:30: *** unterminated call to function `call': missing `)'. Stop. + +| But line 30 only contains ``@$(call targetinfo, $@)`` and it seems all + right. What does it mean? +| A: Yes, this error message is confusing. But it usually only means + that you should check the following (!) lines for missing backslashes + (line separators). + +| Q: I got a message similar to “package is empty. not + generating.” What does it mean? +| A: The ’ipkg’ tool was advised to generate a new ipkg-packet, but the + folder was empty. Sometime it means a typo in the package name when + the install\_copy macro was called. Ensure all these macros are using + the same package name. Or did you disable a menuentry and now nothing + will be installed? + +| Q: How do I download all required packages at once? +| A: Run this command prior the build: + +:: + + > ptxdist make get + +This starts to download all required packages in one run. It does +nothing if the archives are already present in the source path. (run +“PTXdist setup” first). + +| Q: I want to backup the source archives my PTXdist project relys on. + How can I find out what packages my project requires to build? +| A: First build your PTXdist project completely and then run the + following command: + +:: + + > ptxdist export_src + +It copies all archives from where are your source archives stored to + which can be your backup media. + +| Q: To avoid building the OSELAS toolchain on each development host, I + copied it to another machine. But on this machine I cannot build any + BSP with this toolchain correctly. All applications on the target are + failing to start due to missing libraries. +| A: This happens when the toolchain was copied without regarding to + retain links. There are archive programs around that convert links + into real files. When you are using such programs to create a + toolchain archive this toolchain will be broken after extracting it + again. Solution: Use archive programs that retain links as they are + (tar for example). Here an example for a broken toolchain: + +:: + + $ ll `find . -name "libcrypt*"` + -rwxr-xr-x 1 mkl ptx 55K 2007-07-25 14:54 ./lib/libcrypt-2.5.so* + -rwxr-xr-x 1 mkl ptx 55K 2007-07-25 14:54 ./lib/libcrypt.so.1* + -rw-r--r-- 1 mkl ptx 63K 2007-07-25 14:54 ./usr/lib/libcrypt.a + -rw-r--r-- 1 mkl ptx 64K 2007-07-25 14:54 ./usr/lib/libcrypt_p.a + -rwxr-xr-x 1 mkl ptx 55K 2007-07-25 14:54 ./usr/lib/libcrypt.so* + +And in contrast, this one is intact: + +:: + + $ ll `find . -name "libcrypt*"` + -rwxr-xr-x 1 mkl ptx 55K 2007-11-03 13:30 ./lib/libcrypt-2.5.so* + lrwxrwxrwx 1 mkl ptx 15 2008-02-20 14:52 ./lib/libcrypt.so.1 -> libcrypt-2.5.so* + -rw-r--r-- 1 mkl ptx 63K 2007-11-03 13:30 ./usr/lib/libcrypt.a + -rw-r--r-- 1 mkl ptx 64K 2007-11-03 13:30 ./usr/lib/libcrypt_p.a + lrwxrwxrwx 1 mkl ptx 23 2008-02-20 14:52 ./usr/lib/libcrypt.so -> ../../lib/libcrypt.so.1* + +Q: I followed the instructions how to integrate my own plain source +project into PTXdist. But when I try to build it, I get: + +:: + + extract: archive=/path/to/my/sources + extract: dest=/path/to/my/project/build-target + Unknown format, cannot extract! + +| But the path exists! +| A: PTXdist interprets a ``file://`` (two slashes) in the URL as a + project related relative path. So it searches only in the current + project for the given path. Only ``file:///`` (three slashes) will + force PTXdist to use the path as an absolute one. This means: + ``file://bla/blub`` will be used as ``./bla/blub`` and + ``file:///friesel/frasel`` as ``/friesel/frasel``. + +| Q: I want to use more than one kernel revision in my BSP. How can I + avoid maintaining one ptxconfig per kernel? +| A: One solution could be to include the kernel revision into the name + of the kernel config file. Instead of the default kernelconfig.target + name you should use ``kernelconfig-.target``. In the kernel + config file menu entry you should enter + ``kernelconfig-$PTXCONF_KERNEL_VERSION.target``. Whenever you change + the linux kernel Version menu entry now, this will ensure using a + different kernel config file, too. + +| Q: I’m trying to use a JAVA based package in PTXdist. But compiling + fails badly. Does it ever work at Pengutronix? +| A: This kind of packages only build correctly when an original SUN VM + SDK is used. Run PTXdist setup and point the Java SDK menu entry to + the installation path of your SUN JAVA SDK. + +Q: I made a new project and everythings seems fine. But when I start my +target with the root filesystem generated by PTXdist, it fails with: + +:: + + cannot run '/etc/init.d/rcS': No such file or directory + +A: The error message is confusing. But this script needs ``/bin/sh`` to +run. Most of the time this message occures when ``/bin/sh`` does not +exists. Did you enable it in your busybox configuration? + +Q: I have created a path for my source archives and try to make PTXdist +use it. But whenever I run PTXdist now it fails with the following error +message: + +:: + + /usr/local/bin/ptxdist: archives: command not found + +A: In this case the path was ``$HOME/source archives`` which includes a +whitespace in the name of the directory to store the source archives in. +Handling directory or filenames with whitespaces in applications isn’t +trivial and also PTXdist suffers all over the place from this issue. The +only solution is to avoid whitespaces in paths and filenames. + +Q: I have adapted my own rule file’s targetinstall stage, but PTXdist +does not install the files. A: Check if the closing @$(call +install\_finish, [...]) is present at the end of the targetinsall stage. +If not, PTXdist will not complete this stage. + diff --git a/doc/dev_manual.rst b/doc/dev_manual.rst new file mode 100644 index 000000000..1305174c1 --- /dev/null +++ b/doc/dev_manual.rst @@ -0,0 +1,1324 @@ +PTXdist Developer’s Manual +========================== + +This chapter shows all (or most) of the details of how PTXdist works. + +|image| + +- where are the files stored that PTXdist uses when building packages + +- how patching works + +- where is PTXdist fetching a package’s runtime configuration files + from + +- how to control a package’s build stages + +- how to add new packages + +PTXdist’s directory hierarchy +----------------------------- + +Note: Referenced directories are meant relative to the PTXdist main +installation location (if not otherwise stated). If not configured +differently, this main path is ``/usr/local/lib/ptxdist-2015.03.0`` + +Rule Files +~~~~~~~~~~ + +When building a single package, PTXdist needs the information on how to +handle the package, i.e. on how to get it from the source up to what the +target needs at runtime. This information is provided by a rule file per +package. + +PTXdist collects all rule files in its ``rules/`` directory. Whenever +PTXdist builds something, all these rule files are scanned at once. +These rule files are global rule files, valid for all projects. PTXdist +uses a mechanism to be able to add or replace specific rule files on a +per project base. If a ``rules/`` directory exists in the current +project, its content is scanned too. These project local rule files are +used in addition to the global rule files or – if they are using the +same name as a global rule file – **replacing** the global rule file. + +The replacing mechanism can be used to extend or adapt packages for +specific project requirements. Or it can be used for bug fixing by +backporting rule files from more recent PTXdist revisions to projects +that are stuck to an older PTXdist revision for maintenance only. + +Patch Series +~~~~~~~~~~~~ + +There are many packages in the wild that are not cross build aware. They +fail compiling some files, use wrong include paths or try to link +against host libraries. To be sucessful in the embedded world, these +types of failures must be fixed. If required, PTXdist provides such +fixes per package. They are organized in *patch series* and can be found +in the ``patches/`` directory within a subdirectory using the same name +as the package itself. + +PTXdist uses the utility ``patch`` or ``quilt`` to apply an existing +patch series after extracting the archive. So, every patch series +contains a set of patches and one ``series`` file to define the order in +which the patches must be applied. + +Note: Patches can be compressed. + +Besides the ``patches/`` directory at the main installation location, +PTXdist searches two additional locations for a patch series for the +package in question. + +One location is the project’s currently used platform directory. If the +currently used platform is located in ``configs/``, PTXdist searches in +``./configs//patches/``. + +If no patch series was found in the platform directory, the next +location PTXdist it searches for a patch series is the main project +directory in ``./patches/``. + +If both project local locations do not provide a patch series for the +specific package, PTXdist falls back to the ``patches/`` directory at +its main installation location. + +This search order can be used to use specific patch series for specific +cases. + +- platfom specific + +- project specific + +- common case + +- bug fixing + +The *bug fixing* case is used in accordance to a replacement of a rule +file. If this was done due to a backport, and the more recent PTXdist +revision does not only exchange the rule file but also the patch series, +this mechanism ensures that both relevant parts can be updated in the +project. + +Runtime Configuration +~~~~~~~~~~~~~~~~~~~~~ + +Many packages are using runtime configuration files along with their +executables and libraries. PTXdist provides default configuration files +for the most common cases. These files can be found in the +``generic/etc`` directory and they are using the same names as the ones +at runtime (and their install directory on the target side will also be +``/etc``). + +But some of these default configuration files are empty, due to the +absence of a common case. The project must provide replacements of these +files with a more useful content in every case where the (empty) default +one does not meet the target’s requirements. + +PTXdist first searches the project local ``./projectroot/etc`` directory +for a specific configuration file and falls back to use the default one +if none exists locally. + +A popular example is the configuration file ``/etc/fstab``. The default +one coming with PTXdist works for the most common cases. But if our +project requires a special setup, we can just copy the default one to +the local ``./projectroot/etc/fstab``, modify it and we are done. The +next time PTXdist builds the root filesystem it will use the local +``fstab`` instead of the global (default) one. + +Adding new Packages +------------------- + +PTXdist provides a huge amount of applications sufficient for the most +embedded use cases. But there is still need for some fancy new packages. +This section describes the steps and the background on how to integrate +new packages into the project. + +At first a summary about possible application types which PTXdist can +handle: + +- **host type**: This kind of package is built to run on the build + host. Most of the time such a package is needed if another + target-relevant package needs to generate some data. For example the + *glib* package depends on its own to create some data. But if it is + compiled for the target, it can’t do so. That’s why a host glib + package is required to provide these utilities runnable on the build + host. It sounds strange to build a host package, even if on the build + host such utilities are already installed. But this way ensures that + there are no dependencies regarding the build host system. + +- **target type**: This kind of package is built for the target. + +- **cross type**: This kind of package is built for the build host, but + creates architecture specific data for the target. + +- **klibc type**: This kind of package is built against klibc to be + part of an initramfs for the target. + +- **src-autoconf-prog**: This kind of package is built for the target. + It is intended for development, as it does not handle a released + archive but a plain source project instead. Creating such a package + will also create a small autotools based source template project on + demand to give the developer an easy point to start. This template is + prepared to build a single executable program. + +- **src-autoconf-lib**: This kind of package is built for the target. + It is intended for development, as it does not handle a released + archive but a plain source project instead. Creating such a package + will also create a small autotools/libtool based source template + project on demand to give the developer an easy point to start. This + template is prepared to build a single shared library. For further + details refer section + [adding:sub:`s`\ rc\ :sub:`a`\ utoconf\ :sub:`l`\ ib] + +- **src-autoconf-proglib**: This kind of package is built for the + target. It is intended for development, as it does not handle a + released archive but a plain source project instead. Creating such a + package will also create a small autotools/libtool based template + project on demand to give the developer an easy point to start. This + template is prepared to build a single shared library and a single + executable program. The program will be linked against the shared + library. + +- **file**: This kind of package is intended to add a few simple files + into the build process. We assume these files do not need any + processing, they are ready to use and must only be present in the + build process or at runtime (HTML files for example). Refer to the + section [chap:adding:sub:`f`\ iles] for further details on how to use + it. + +- **src-make-prog**: This kind of package is built for the target. It’s + intended for development, as it does not handle a released archive + but a plain source project instead. Creating such a package will also + create a simple makefile-based template project the developer can use + as a starting point for development. + +- **src-cmake-prog**: This kind of package is built for the target. + It’s intended for developments based on the *cmake* buildsystem. + Various projects are using *cmake* instead of *make* and can be built + with this package type. PTXdist will prepare it to compile sources in + accordance to the target libraries and their settings. Creating such + a package will also create a simple template project to be used as a + starting point for development. + +- **src-qmake-prog**: This kind of package is built for the target. + It’s intended for developments based on the *qmake* buildsystem. If + the developer is going to develop a QT based application, this rule + is prepared to compile sources in accordance to the target libraries + and their settings. Creating such a package will also create a simple + template project to be used as a starting point for development. + +- **font**: This package is a helper to add X font files to the root + filesystem. This package does not create an additional IPKG, instead + it adds the font to the existing font IPGK. This includes the + generation of the directory index files, required by the Xorg + framework to recognize the font file. + +- **src-linux-driver**: This kind of package builds an out of tree + kernel driver. It also creates a driver template to give the + developer an easy point to start. + +Rule File Creation +~~~~~~~~~~~~~~~~~~ + +To create such a new package, we create a project local ``rules/`` +directory first. Then we run + +:: + + $ ptxdist newpackage |\textless{}package type\textgreater{}| + +If we omit the <``package type``\ >, PTXdist will list all available +package types. + +In our first example, we want to add a new target type archive package. +When running the + +:: + + $ ptxdist newpackage target + +command, PTXdist asks a few questions about this package. This +information is the basic data PTXdist must know about the package. + +:: + + ptxdist: creating a new 'target' package: + + ptxdist: enter package name.......: foo + ptxdist: enter version number.....: 1.1.0 + ptxdist: enter URL of basedir.....: http://www.foo.com/download/src + ptxdist: enter suffix.............: tar.gz + ptxdist: enter package author.....: My Name + ptxdist: enter package section....: project_specific + +What we have to answer: + +- **package name**: As this kind of package handles a source archive, + the correct answer here is the basename of the archive’s file name. + If its full name is ``foo-1.1.0.tar.gz``, then ``foo`` is the + basename to enter here. + +- **version number**: Most source archives are using a release or + version number in their file name. If its full name is + ``foo-1.1.0.tar.gz``, then ``1.1.0`` is the version number to enter + here. + +- **URL of basedir**: This URL tells PTXdist where to download the + source archive from the web (if not already done). If the full URL to + download the archive is + ``http://www.foo.com/download/src/foo-1.1.0.tar.gz``, the basedir + part ``http://www.foo.com/download/src`` is to be entered here. + +- **suffix**: Archives are using various formats for distribution. + PTXdist uses the *suffix* entry to select the matching extraction + tool. If the archive’s full name is ``foo-1.1.0.tar.gz``, then + ``tar.gz`` is the suffix to enter here. + +- **package author**: If we intend to contribute this new package to + PTXdist mainline, we should add our name here. This name will be used + in the copyright note of the rule file and will also be added to the + generated ipkg. When you run ``ptxdist setup`` prior to this call, + you can enter your name and your email address, so PTXdist will use + it as the default (very handy if you intend to add many new + packages). + +- **package section**: We can enter here the menu section name where + our new package menu entry should be listed. In the first step we can + leave the default name unchanged. It’s a string in the menu file + only, so changing it later on is still possible. + +Make it Work +~~~~~~~~~~~~ + +Generating the rule file is only one of the required steps to get a new +package. The next steps to make it work are to check if all stages are +working as expected and to select the required parts to get them +installed in the target root filesystem. Also we must find a reasonable +location where to add our new menu entry to configure the package. + +The generated skeleton starts to add the new menu entry in the main +configure menu (if we left the section name unchanged). Running +``ptxdist menuconfig`` will show it on top of all other menus entries. + +To be able to implement and test all the other required steps for adding +a new package, we first must enable the package for building. (Fine +tuning the menu can happen later on.) + +The rule file skeleton still lacks some important information. Let’s +take a look into some of the top lines of the generated rule file +``./rules/foo.make``: + +:: + + FOO_VERSION := 1.1.0 + FOO_MD5 := + FOO := foo-$(FOO_VERSION) + FOO_SUFFIX := tar.gz + FOO_URL := http://www.foo.com/download/src/$(FOO).$(FOO_SUFFIX) + FOO_SOURCE := $(SRCDIR)/$(FOO).$(FOO_SUFFIX) + FOO_DIR := $(BUILDDIR)/$(FOO) + FOO_LICENSE := unknown + +We can find these lines with different content in most or all of the +other rule files PTXdist comes with. Up to the underline character is +always the package name and after the underline character is always +PTXdist specific. What does it mean: + +- \*\_VERSION brings in the version number of the release and is used + for the download and IPKG/OPKG package generation. + +- \*\_MD5 to be sure the correct package has been downloaded, PTXdist + checks the given MD5 sum against the archive content. If both sums do + not match, PTXdist rejects the archive and fails the currently + running build. + +- \*\_SUFFIX defines the archive type, to make PTXdist choosing the + correct extracting tool. + +- \*\_URL defines the full qualified URL into the web for download. If + alternative download locations are known, they can be listed in this + variable, delimiter character is the space. + +- \*\_DIR points to the directory this package will be build later on + by PTXdist + +- \*\_LICENSE enables the user to get a list of licenses she/he is + using in her/his project (licenses of the enabled packages). + +After enabling the menu entry, we can start to check the *get* and +*extract* stages, calling them manually one after another. + +Note: The shown commands below expect that PTXdist downloads the +archives to a global directory named ``global_src``. This is not the +default setting, but we recommend to use a global directory to share all +archives between PTXdist based projects. Advantage is every download +happens only once. Refer to the ``setup`` command PTXdist provides. + +:: + + $ ptxdist get foo + + --------------------------- + target: foo-1.1.0.tar.gz + --------------------------- + + --2009-12-21 10:54:45-- http://www.foo.com/download/src/foo-1.1.0.tar.gz + Length: 291190 (284K) [application/x-gzip] + Saving to: `/global_src/foo-1.1.0.tar.gz.XXXXOGncZA' + + 100%[======================================>] 291,190 170K/s in 1.7s + + 2009-12-21 10:54:48 (170 KB/s) - `/global_src/foo-1.1.0.tar.gz' saved [291190/291190] + +This command should start to download the source archive. If it fails, +we should check our network connection, proxy setup or if the given URL +in use is correct. + +Note: Sometimes we do not know the content of all the other variables in +the rule file. To get an idea what content a variable has, we can ask +PTXdist about it: + +:: + + $ ptxdist print FOO_URL + http://www.foo.com/download/src/foo-1.1.0.tar.gz + +The next step would be to extract the archive. But as PTXdist checks the +MD5 sum in this case, this step will fail, because the ``FOO_MD5`` +variable is still empty. Let’s fill it: + +:: + + $ md5sum /global_src/foo-1.1.0.tar.gz + 9a09840ab775a139ebb00f57a587b447 + +This string must be assigned to the FOO\_MD5 in our new ``foo.make`` +rule file: + +:: + + FOO_MD5 := 9a09840ab775a139ebb00f57a587b447 + +We are now prepared for the next step: + +:: + + $ ptxdist extract foo + + ----------------------- + target: foo.extract + ----------------------- + + extract: archive=/global_src/foo-1.1.0.tar.gz + extract: dest=/home/jbe/my_new_prj/build-target + PATCHIN: packet=foo-1.1.0 + PATCHIN: dir=/home/jbe/my_new_prj/build-target/foo-1.1.0 + PATCHIN: no patches for foo-1.1.0 available + Fixing up /home/jbe/my_new_prj/build-target/foo-1.1.0/configure + finished target foo.extract + +In this example we expect an autotoolized source package. E.g. to +prepare the build, the archive comes with a ``configure`` script. This +is the default case for PTXdist. So, there is no need to modify the rule +file and we can simply run: + +:: + + $ ptxdist prepare foo + + ----------------------- + target: foo.prepare + ----------------------- + + [...] + + checking build system type... i686-host-linux-gnu + checking host system type... |\ptxdistCompilerName{}| + checking whether to enable maintainer-specific portions of Makefiles... no + checking for a BSD-compatible install... /usr/bin/install -c + checking whether build environment is sane... yes + checking for a thread-safe mkdir -p... /bin/mkdir -p + checking for gawk... gawk + checking whether make sets $(MAKE)... yes + checking for |\ptxdistCompilerName{}|-strip... |\ptxdistCompilerName{}|-strip + checking for |\ptxdistCompilerName{}|-gcc... |\ptxdistCompilerName{}|-gcc + checking for C compiler default output file name... a.out + + [...] + + configure: creating ./config.status + config.status: creating Makefile + config.status: creating ppa_protocol/Makefile + config.status: creating config.h + config.status: executing depfiles commands + finished target foo.prepare + +At this stage things can fail: + +- A wrong or no MD5 sum was given + +- The ``configure`` script is not cross compile aware + +- The package depends on external components (libraries for example) + +| If the ``configure`` script is not cross compile aware, we are out of + luck. We must patch the source archive in this case to make it work. + Refer to section [sect:configure:sub:`r`\ ebuild] on how to use + PTXdist’s features to simplify this task. +| If the package depends on external components, these components might + be already part of PTXdist. In this case we just have to add this + dependency into the menu file and we are done. But if PTXdist cannot + fulfill this dependency, we also must add it as a separate package + first. + +If the *prepare* stage has finished successfully, the next step is to +compile the package. + +:: + + $ ptxdist compile foo + + ----------------------- + target: foo.compile + ----------------------- + + make[1]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make all-recursive + make[2]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make[3]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + + [...] + + make[3]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make[2]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make[1]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + finished target foo.compile + +At this stage things can fail: + +- The build system is not cross compile aware (it tries to execute just + created target binaries for example) + +- The package depends on external components (libraries for example) + not detected by ``configure`` + +- Sources are ignoring the endianess of some architectures or using + header files from the build host system (from ``/usr/include`` for + example) + +- The linker uses libraries from the build host system (from + ``/usr/lib`` for example) by accident + +In all of these cases we must patch the sources to make them work. Refer +to section [sect:patching:sub:`p`\ ackages] on how to use PTXdist’s +features to simplify this task. + +In this example we expect the best case: Everything went fine, even for +cross compiling. So, we can continue with the next stage: *install* + +:: + + $ ptxdist install foo + + ----------------------- + target: foo.install + ----------------------- + + make[1]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make[2]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make[3]: Entering directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + test -z "/usr/bin" || /bin/mkdir -p "/home/jbe/my_new_prj/build-target/foo-1.1.0/usr/bin" + /usr/bin/install -c 'foo' '/home/jbe/my_new_prj/build-target/foo-1.1.0/usr/bin/foo' + make[3]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make[2]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + make[1]: Leaving directory `/home/jbe/my_new_prj/build-target/foo-1.1.0' + finished target foo.install + + ---------------------------- + target: foo.install.post + ---------------------------- + + finished target foo.install.post + +This *install* stage does not install anything to the target root +filesystem. It is mostly intended to install libraries and header files +other programs should link against later on. + +The last stage – *targetinstall* – is the one that defines the package’s +components to be forwarded to the target’s root filesystem. Due to the +absence of a generic way, this is the task of the developer. So, at this +point of time we must run our favourite editor again and modify our new +rule file ``./rules/foo.make``. + +The skeleton for the *targetinstall* stage looks like this: + +:: + + # ---------------------------------------------------------------------------- + # Target-Install + # ---------------------------------------------------------------------------- + + $(STATEDIR)/foo.targetinstall: + @$(call targetinfo) + + @$(call install_init, foo) + @$(call install_fixup, foo,PACKAGE,foo) + @$(call install_fixup, foo,PRIORITY,optional) + @$(call install_fixup, foo,VERSION,$(FOO_VERSION)) + @$(call install_fixup, foo,SECTION,base) + @$(call install_fixup, foo,AUTHOR,"My Name ") + @$(call install_fixup, foo,DEPENDS,) + @$(call install_fixup, foo,DESCRIPTION,missing) + + @$call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foobar, /dev/null) + + @$(call install_finish, foo) + @$(call touch) + +The “header” of this stage defines some information IPKG needs. The +important part that we must modify is the call to the ``install_copy`` +macro (refer to section [sect:reference:sub:`m`\ acros] for more details +about this kind of macros). This call instructs PTXdist to include the +given file (with PID, UID and permissions) into the IPKG, which means to +install this file to the target’s root filesystem. + +From the previous *install* stage we know this package installs an +executable called ``foo`` to location ``/usr/bin``. We can do the same +for our target by changing the *install\_copy* line to: + +:: + + @$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/foo) + +To check it, we just run: + +:: + + $ ptxdist targetinstall foo + + ----------------------------- + target: foo.targetinstall + ----------------------------- + + install_init: preparing for image creation... + install_init: @ARCH@ -> i386 ... done + install_init: preinst not available + install_init: postinst not available + install_init: prerm not available + install_init: postrm not available + install_fixup: @PACKAGE@ -> foo ... done. + install_fixup: @PRIORITY@ -> optional ... done. + install_fixup: @VERSION@ -> 1.1.0 ... done. + install_fixup: @SECTION@ -> base ... done. + install_fixup: @AUTHOR@ -> "My Name " ... done. + install_fixup: @DESCRIPTION@ -> missing ... done. + install_copy: + src=/home/jbe/my_new_prj/build-target/foo-1.1.0/foo + dst=/usr/bin/foo + owner=0 + group=0 + permissions=0755 + xpkg_finish: collecting license (unknown) ... done. + xpkg_finish: creating ipkg package ... done. + finished target foo.targetinstall + + ---------------------------------- + target: foo.targetinstall.post + ---------------------------------- + + finished target foo.targetinstall.post + +After this command, the target’s root filesystem contains a file called +``/usr/bin/foo`` owned by root, its group is also root and everyone has +execution permissions, but only the user root has write permissions. + +| One last task of this port is still open: A reasonable location for + the new menu entry in PTXdist’s menu hierarchy. PTXdist arranges its + menus on the meaning of each package. Is it a network related tool? Or + a scripting language? Or a graphical application? +| Each of these global meanings have their own submenu, where we can add + our new entry to. We just have to edit the head of our new menu file + ``./rules/foo.in`` to add it to a specific global menu. If our new + package is a network related tool, the head of the menu file should + look like: + +:: + + ## SECTION=networking + +We can grep through the other menu files from the PTXdist main +installation ``rules/`` directory to get an idea what section names are +available: + +:: + + rules/ @\$@ find . -name \*.in | xargs grep "## SECTION" + ./acpid.in:## SECTION=shell_and_console + ./alsa-lib.in:## SECTION=system_libraries + ./alsa-utils.in:## SECTION=multimedia_sound + ./apache2.in:## SECTION=networking + ./apache2_mod_python.in:## SECTION=networking + [...] + ./klibc-module-init-tools.in:## SECTION=initramfs + ./xkeyboard-config.in:## SECTION=multimedia_xorg_data + ./xorg-app-xev.in:## SECTION=multimedia_xorg_app + ./xorg-app-xrandr.in:## SECTION=multimedia_xorg_app + ./host-eggdbus.in:## SECTION=hosttools_noprompt + ./libssh2.in:## SECTION=networking + +Porting a new package to PTXdist is finished now. + +To check it right away, we simply run these two commands: + +:: + + $ ptxdist clean foo + rm -rf /home/jbe/my_new_prj/state/foo.* + rm -rf /home/jbe/my_new_prj/packages/foo_* + rm -rf /home/jbe/my_new_prj/build-target/foo-1.1.0 + $ ptxdist targetinstall foo + + [...] + +Advanced Rule Files +~~~~~~~~~~~~~~~~~~~ + +The previous example on how to create a rule file sometimes works as +shown above. But most of the time source archives are not that simple. +In this section we want to give the user a more detailed selection how +the package will be built. + +Adding Static Configure Parameters +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``configure`` scripts of various source archives provide additional +parameters to enable or disable features, or to configure them in a +specific way. + +We assume the ``configure`` script of our ``foo`` example (refer to +section [sect:foo:sub:`e`\ xample]) supports two additional parameters: + +- **--enable-debug**: Make the program more noisy. It’s disabled by + default. + +- **--with-bar**: Also build the special executable **bar**. Building + this executable is also disabled by default. + +We now want to forward these options to the ``configure`` script when it +runs in the *prepare* stage. To do so, we must again open the rule file +with our favourite editor and navigate to the *prepare* stage entry. + +PTXdist uses the variable ``FOO_AUTOCONF`` as the list of parameters to +be given to ``configure``. + +Currently this variable is commented out and defined to: + +:: + + # FOO_AUTOCONF := $(CROSS_AUTOCONF_USR) + +The variable ``CROSS_AUTOCONF_USR`` is predefined by PTXdist and +contains all basic parameters to instruct ``configure`` to prepare for a +**cross** compile environment. + +To use the two additional mentioned ``configure`` parameters, we comment +in this line and supplement this expression as follows: + +:: + + FOO_AUTOCONF := $(CROSS_AUTOCONF_USR) \ + --enable-debug \ + --with-bar + +Note: We recommend to use this format with each parameter on a line of +its own. This format is easier to read and a diff shows more exactly any +change. + +To do a fast check if this addition was successful, we run: + +:: + + $ ptxdist print FOO_AUTOCONF + --prefix=/usr --sysconfdir=/etc --host=|\ptxdistCompilerName{}| --build=i686-host-linux-gnu --enable-debug --with-bar + +Note: It depends on the currently selected platform and its architecture +what content this variable will have. The content shown above is an +example for an target. + +Or re-build the package with the new settings: + +:: + + $ ptxdist drop foo prepare + $ ptxdist targetinstall foo + +Adding Dynamic Configure Parameters +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Sometimes it makes sense to add this kind of parameters on demand only; +especially a parameter like ``--enable-debug``. To let the user decide +if this parameter is to be used or not, we must add a menu entry. So, +let’s expand our menu. Here is its current content: + +:: + + ## SECTION=project_specific + + config FOO + tristate + prompt "foo" + help + FIXME + +We’ll add two menu entries, one for each optional parameter we want to +add on demand to the ``configure`` parameters: + +:: + + ## SECTION=project_specific + + config FOO + tristate + prompt "foo" + help + FIXME + + if FOO + config FOO_DEBUG + bool + prompt "add debug noise" + + config FOO_BAR + bool + prompt "build bar" + + endif + +Note: To extend the base name by a suboption name as a trailing +component gives PTXdist the ability to detect a change in the package’s +settings to force its rebuild. + +To make usage of the new menu entries, we must check them in the rule +file and add the correct parameters: + +:: + + # + # autoconf + # + FOO_AUTOCONF := $(CROSS_AUTOCONF_USR) + + ifdef PTXCONF_FOO_DEBUG + FOO_AUTOCONF += --enable-debug + else + FOO_AUTOCONF += --disable-debug + endif + + ifdef PTXCONF_FOO_BAR + FOO_AUTOCONF += --with-bar + else + FOO_AUTOCONF += --without-bar + endif + +Please note the trailing ``PTXCONF_`` for each define. While Kconfig is +using ``FOO_BAR``, the rule file must use ``PTXCONF_FOO_BAR`` instead. + +It is a good practice to add both settings, e.g. ``--disable-debug`` +even if this is the default case. Sometimes ``configure`` tries to guess +something and the binary result might differ depending on the build +order. For example some kind of package would also build some X related +tools, if X libraries are found. In this case it depends on the build +order, if the X related tools are built or not. All the autocheck +features are problematic here. So, if we do not want ``configure`` to +guess its settings we **must disable everything we do want**. + +Since every optional parameter adds four lines of code to the rule +files, PTXdist provides some shortcuts to handle it. Refer to section +[ref:sub:`p`\ aram\ :sub:`m`\ acros] for further details. + +With these special macros in use, the file content shown above looks +much simpler: + +:: + + # + # autoconf + # + FOO_AUTOCONF := $(CROSS_AUTOCONF_USR) \ + $(call ptx/endis, PTXCONF_FOO_DEBUG)-debug \ + $(call ptx/wwo, PTXCONF_FOO_BAR)-bar + +If some parts of a package are built on demand only, they must also be +installed on demand only. Besides the *prepare* stage, we also must +modify our *targetinstall* stage: + +:: + + + @$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/foo) + + ifdef PTXCONF_FOO_BAR + @$(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/bar, /usr/bin/bar) + endif + + @$(call install_finish, foo) + @$(call touch) + + [...] + +Now we can play with our new menu entries and check if they are working +as expected: + +:: + + $ ptxdist menuconfig + $ ptxdist targetinstall foo + +Whenever we change a *FOO* related menu entry, PTXdist should detect it +and re-build the package when a new build is started. + +Managing External Compile Time Dependencies +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +While running the prepare stage, it could happen that it fails due to a +missing external dependency. + +For example: + +:: + + checking whether zlib exists....failed + +In this example, our new package depends on the compression library +*zlib*. PTXdist comes with a target *zlib*. All we need to do in this +case is to declare that our new package *foo* depends on *zlib*. This +kind of dependencies is managed in the menu file of our new package by +simply adding the ``select ZLIB`` line. After this addition our menu +file looks like: + +:: + + ## SECTION=project_specific + + config FOO + tristate + select ZLIB + prompt "foo" + help + FIXME + + if FOO + config FOO_DEBUG + bool + prompt "add debug noise" + + config FOO_BAR + bool + prompt "build bar" + + endif + +PTXdist now builds the *zlib* first and our new package thereafter. + +Managing External Compile Time Dependencies on Demand +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +It is good practice to add only those dependecies that are really +required for the current configuration of the package. If the package +provides the features *foo* and *bar* and its ``configure`` provides +switches to enable/disable them independently, we can also add +dependencies on demand. Let’s assume feature *foo* needs the compression +library *libz* and *bar* needs the XML2 library *libxml2*. These +libraries are only required at runtime if the correspondig feature is +enabled. To add these dependencies on demand, the menu file looks like: + +:: + + ## SECTION=project_specific + + config FOO + tristate + select ZLIB if FOO_FOO + select LIBXML2 if FOO_BAR + prompt "foo" + help + FIXME + + if FOO + config FOO_DEBUG + bool + prompt "add debug noise" + + config FOO_FOO + bool + prompt "build foo" + + config FOO_BAR + bool + prompt "build bar" + + endif + +Do not add these ``select`` statements to the correspondig menu entry. +They must belong to the main menu entry of the package to ensure that +the calculation of the dependencies between the packages is done in a +correct manner. + +Managing External Runtime Dependencies +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Some packages are building all of their components and also installing +them into the target’s sysroot. But only their *targetinstall* stage +decides which parts are copied to the root filesystem. So, compiling and +linking of our package will work, because everything required is found +in the target’s sysroot. + +In our example there is a hidden dependency to the math library +``libm``. Our new package was built successfully, because the linker was +able to link our binaries against the ``libm`` from the toolchain. But +in this case the ``libm`` must also be available in the target’s root +filesystem to fulfil the runtime dependency: We have to force PTXdist to +install ``libm``. ``libm`` is part of the *glibc* package, but is not +installed by default (to keep the root filesystem small). So, it **does +not** help to select the ``GLIBC`` symbol, to get a ``libm`` at runtime. + +The correct solution here is to add a ``select LIBC_M`` to our menu +file. With all the additions above it now looks like: + +:: + + ## SECTION=project_specific + + config FOO + tristate + select ZLIB if FOO_FOO + select LIBXML2 if FOO_BAR + select LIBC_M + prompt "foo" + help + FIXME + + if FOO + config FOO_DEBUG + bool + prompt "add debug noise" + + config FOO_FOO + bool + prompt "build foo" + + config FOO_BAR + bool + prompt "build bar" + + endif + +Note: There are other packages around, that do not install everything by +default. If our new package needs something special, we must take a look +into the menu of the other package how to force the required components +to be installed and add the corresponding ``selects`` to our own menu +file. In this case it does not help to enable the required parts in our +project configuration, because this has no effect on the build order! + +Managing Non Autotool Packages +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Many packages are still coming with a plain ``Makefile``. The user has +to adapt it to make it work in a cross compile environment as well. +PTXdist can also handle this kind of packages. We only have to specifiy +a special *prepare* and *compile* stage. + +Such packages often have no special need for any kind of preparation. We +can omit this stage by defining this empty rule: + +:: + + $(STATEDIR)/foo.prepare: + @$(call targetinfo) + @$(call touch) + +To compile the package, we can use ``make``\ ’s feature to overwrite +variables used in the ``Makefile``. With this feature we can still use +the original ``Makefile`` but with our own (cross compile) settings. + +Most of the time the generic compile rule can be used, only a few +settings are required. To use only ``make`` instead of the autotools, we +must instruct PTXdist to not use them by defining: + +:: + + FOO_CONF_TOOL := NO + +``make`` will be called in this case with: + +``cd $(FOO_DIR) && $(FOO_MAKE_ENV) $(MAKE) $(FOO_MAKE_OPT)`` + +So, in the rule file only the two variables ``FOO_MAKE_ENV`` and +``FOO_MAKE_OPT`` must be set, to forward the required settings to the +package’s buildsystem. If the package cannot be built in parallel, we +can also add the ``FOO_MAKE_PAR := NO``. ``YES`` is the default. + +Note: *FOO* is still the name of our example package. It must be +replaced by the real package name. + +Patching Packages +~~~~~~~~~~~~~~~~~ + +There can be various reasons why a package must be patched: + +- Package is broken for cross compile environments + +- Package is broken within a specific feature + +- Package is vulnerable and needs some fixes + +- or anything else (this case is the most common one) + +PTXdist handles patching automatically. After extracting the archive, +PTXdist checks for the existence of a patch directory with the same name +as the package. If our package’s name is ``foo-1.1.0``, PTXdist searches +for patches in: + +#. project (``./patches/foo-1.1.0``) + +#. platform (``./configs//patches/foo-1.1.0``) + +#. ptxdist (``/patches/foo-1.1.0``) + +The patches from the first location found are used. Note: Due to this +search order, a PTXdist project can replace global patches from the +PTXdist installation. This can be useful if a project sticks to a +specific PTXdist revision but fixes from a more recent revision of +PTXdist should be used. + +Creating Patches for a Package +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PTXdist uses the utilities ``git``, *patch* or *quilt* to work with +patches or patch series. We recommend *quilt*, as it can manage patch +series in a very easy way. For this manual we assume *quilt* is +installed on the build host. + +Creating a Patch Series for a Package +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To create a patch series for the first time, we can run the following +steps. We are still using our *foo-1.1.0* example package here: + +We create a special directory for the patch series in the local project +directory: + +:: + + $ mkdir -p patches/foo-1.1.0 + +PTXdist expects a ``series`` file in the patch directory and at least +one patch. Otherwise it fails. Due to the fact that we do not have any +patch content yet, we’ll start with a dummy entry in the ``series`` file +and an empty ``patch`` file. + +:: + + $ touch patches/foo-1.1.0/dummy + $ echo dummy > patches/foo-1.1.0/series + +Next is to extract the package (if already done, we must remove it +first): + +:: + + $ ptxdist extract foo + +This will extract the archive and create a symbolic link in the build +directory pointing to our local patch directory. Working this way will +ensure that we do not lose our created patches if we enter +``ptxdist clean foo`` by accident. In our case the patches are still +present in ``patches/foo-1.1.0`` and can be used the next time we +extract the package again. + +All we have to do now is to do the modification we need to make the +package work. We change into the build directory and use *quilt* to +create new patches, add files to respective patches, modify these files +and refresh the patches to save our changes. + +We recommend this way when modifying source files. But this way is +improper when an autotools based buildsystem itself needs modifications. +Refer to section [sect:configure:sub:`r`\ ebuild] on how PTXdist can +handle this special task. + +Adding more Patches to a Package +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If we want to add more patches to an already patched package, we can use +nearly the same way as creating patches for the first time. But if the +patch series comes from the PTXdist main installation, we do not have +write permissions to these directories (do NEVER work on the main +installation directories, NEVER, NEVER, NEVER). Due to the search order +in which PTXdist searches for patches for a specific package, we can +copy the global patch series to our local project directory. Now we have +the permissions to add more patches or modify the existing ones. Also +*quilt* is our friend here to manage the patch series. + +If we think that our new patches are valuable also for others, or they +fix an error, it could be a good idea to send these patches to PTXdist +mainline. + +Modifying Autotoolized Packages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Autotoolized packages are very picky when automatically generated files +get patched. The patch order is very important in this case and +sometimes it even fails and nowbody knows why. + +To improve a package’s autotools-based build system, PTXdist comes with +its own project local autotools to regenerate the autotools template +files, instead of patching them. With this feature, only the template +files must be patched, the required ``configure`` script and the +``Makefile.in`` files are regenerated in the final stages of the +*prepare* step. + +This feature works like the regular patching mechanism. The only +difference is the additional ``autogen.sh`` file in the patch directory. +If it exists and has execution permissions, it will be called after the +package was patched (while the *extract* stage is running). + +Its content depends on developer needs; for the most simple case the +content can be: + +:: + + #!/bin/bash + + aclocal $ACLOCAL_FLAGS + + libtoolize \ + --force \ + --copy + + autoreconf \ + --force \ + --install \ + --warnings=cross \ + --warnings=syntax \ + --warnings=obsolete \ + --warnings=unsupported + +Note: In this way not yet autotoolized package can be autotoolized. We +just have to add the common autotool template files (``configure.ac`` +and ``Makefile.am`` for example) via a patch series to the package +source and the ``autogen.sh`` to the patch directory. + +Adding binary only Files +------------------------ + +Sometimes a few binary files have to be added into the root filesystem. +Or - to be more precise - some files, that do not need to be built in +any way. + +On the other hand, sometimes files should be included that are not +covered by any open source license and so, should not be shipped in the +source code format. + +PTXdist provides more than one way to add such type of data files to the +root filesystem it creates. The examples in this chapter refer our +generic board support package. It comes with an example how to add +binary only files into PTXdist’s build mechanism. + +Old style - single files +~~~~~~~~~~~~~~~~~~~~~~~~ + +The old style to add a simple file is present in PTXdist since its early +days: Just use the ``install_copy`` macro in the *targetinstall* stage +in your own customized rules file. + +:: + + @$(call install_copy, binary_example, 0, 0, 0644, \ + $(PTXDIST_WORKSPACE)/local_src/binary_example/ptx_logo.png, \ + /example/ptx_logo.png) + +The example above is from the file ``rules/binary_inst.make`` from +Pengutronix’s generic BSP. It copies the file ``ptx_logo.png`` from +within the BSP’s directory ``local_src/binary_example`` to target’s root +filesystem. Refer [ref:sub:`i`\ nstall\ :sub:`c`\ opy] for further +information about using the ``install_copy`` macro. + +The disadvantage of this method is: If we want to install more than one +file, we need one call to the ``install_copy`` macro per file. This is +even harder if not only a set of files is to be installed, but a whole +directory tree with files instead. + +New style - using archives +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If a whole tree of files is to be installed, working with a *tar* based +archive could make life easier. In this case the archive itself provides +all the required information the files are needing to be installed in a +correct manner: + +- the file itself and its name + +- the directory structure and the final location of every file in this + structure + +- user and group ID on a per file base + +:: + + @$(call install_archive, binary_example, -, -, \ + $(PTXDIST_WORKSPACE)/local_src/archive_example/pictures.tgz, \ + /) + +The example shown above is from the file ``rules/binary_inst.make`` from +Pengutronix’s generic BSP. It extracts the archive ``pictures.tgz`` from +within the BSP’s directory ``local_src/archive_example`` to target’s +root filesystem. Refer [ref:sub:`i`\ nstall\ :sub:`a`\ rchive] for +further information about using the ``install_archive`` macro. + +Using an archive can be usefull to install parts of the root filesystem +that are not covered by any open source license. Its possible to ship +the binaries within the regular BSP, without the need for their sources. +However it is possible for the customer to re-create everything required +from the BSP to get their target up and running again. + +Another use case for the archive method could be the support for +different development teams. One team provides a software component in +the archive format, the other team does not need to build it but can use +it in the same way than every other software component. + +Creating a Rules File +~~~~~~~~~~~~~~~~~~~~~ + +To get a rules and menu file we can copy the one from the generic BSP, +or we let PTXdist create them for us. + +:: + + $ ptxdist newpackage file + + ptxdist: creating a new 'file' package: + + ptxdist: enter package name.......: my_binfiles + ptxdist: enter version number.....: 1 + ptxdist: enter package author.....: Juergen Borleis + ptxdist: enter package section....: rootfs + +Now two new files are present in the BSP: + +#. ``rules/my_binfiles.in`` The template for the menu + +#. ``rules/my_binfiles.make`` The rules template + +Both files now must be customized to meet our requirements. Due to the +answer *rootfs* to the “``enter package section``” question, we will +find the new menu entry in: + +:: + + Root Filesystem ---> + < > my_binfiles (NEW) + +Enabling this new entry will also run our stages in +``rules/my_binfiles.make`` the next time we enter: + +:: + + $ ptxdist go + diff --git a/doc/environment.rst b/doc/environment.rst new file mode 100644 index 000000000..5d51a2de7 --- /dev/null +++ b/doc/environment.rst @@ -0,0 +1,687 @@ +Getting a working Environment +============================= + +|image| + +Download Software Components +---------------------------- + +In order to follow this manual, some software archives are needed. There +are several possibilities how to get these: either as part of an +evaluation board package or by downloading them from the Pengutronix web +site. + +In order to follow this application note, some software archives are +needed. There are several possibilities how to get these: either as part +of an evaluation board package, or by download from a world wide web +site. + +The central place for OSELAS related documentation is +http://www.oselas.com and http://www.ptxdist.de. These websites provide +all required packages and documentation (at least for software +components which are available to the public). + +In order to build , the following source archives have to be available +on the development host: + +ptxdist-.tar.bz2 + +.tar.gz + +ptxdist-.tar.bz2 + +OSELAS.Toolchain-.tar.bz2 + +If they are not available on the development system yet, it is necessary +to get them. + +The PTXdist- is only required to build the toolchain, while PTXdist- is +required to build the BSP. All PTXdist revisions can co-exist. + +To follow the steps below, the following archive has to be available on +the development host: + +- ``ptxdist-.tar.bz2`` + +PTXdist Installation +-------------------- + +PTXdist is shipped in an archive. This application note provides +information about which archive is required to be downloaded and +installed to get a working environment to build filesystems for their +target systems. + +The PTXdist build system can be used to create a root filesystem for +embedded Linux devices. In order to start development with PTXdist it is +necessary to install the software on the development system. + +This chapter provides information about how to install and configure +PTXdist on the development host. + +Main Parts of PTXdist +~~~~~~~~~~~~~~~~~~~~~ + +The most important software component which is necessary to build an +OSELAS.BSP( ) board support package is the ``ptxdist`` tool. So before +starting any work we’ll have to install PTXdist on the development host. + +PTXdist consists of the following parts: + +The ``ptxdist`` Program: + ``ptxdist`` is installed on the development host during the + installation process. ``ptxdist`` is called to trigger any action, + like building a software packet, cleaning up the tree etc. Usually + the ``ptxdist`` program is used in a *workspace* directory, which + contains all project relevant files. + +A Configuration System: + The config system is used to customize a *configuration*, which + contains information about which packages have to be built and which + options are selected. + +Patches: + Due to the fact that some upstream packages are not bug free + – especially with regard to cross compilation – it is often + necessary to patch the original software. PTXdist contains a + mechanism to automatically apply patches to packages. The patches + are bundled into a separate archive. Nevertheless, they are + necessary to build a working system. + +Package Descriptions: + For each software component there is a “recipe” file, specifying + which actions have to be done to prepare and compile the software. + Additionally, packages contain their configuration sniplet for the + config system. + +Toolchains: + PTXdist does not come with a pre-built binary toolchain. + Nevertheless, PTXdist itself is able to build toolchains, which are + provided by the OSELAS.Toolchain() project. More in-deep information + about the OSELAS.Toolchain() project can be found here: + http://www.pengutronix.de/oselas/toolchain/index_en.html + + Building a toolchain is not part of this application note, refer for + application note “Building Toolchains” instead. + +Board Support Package + This is an optional component, mostly shipped aside with a piece of + hardware. There are various BSP available, some are generic, some + are intended for a specific hardware. + +Download Required Archives +-------------------------- + +Extracting the Sources +~~~~~~~~~~~~~~~~~~~~~~ + +Do the following steps at best in your own home directory ($HOME). You +need root permissions only in the ``make install`` step, and **nowhere** +else. + +To install PTXdist, the archive Pengutronix provides has to be +extracted: + +ptxdist-.tar.bz2 + The PTXdist software itself + +The PTXdist archive has to be extracted into some temporary directory in +order to be built before the installation, for example the ``local/`` +directory in the user’s home. If this directory does not exist, we have +to create it and change into it: + +:: + + $ cd + $ mkdir local + $ cd local + +Next step is to extract the archive: + +:: + + $ tar -xjf ptxdist-|\verbatimcmd\ptxdistVendorVersion{}|.tar.bz2 + +If everything goes well, we now have a PTXdist- directory, so we can +change into it: + +:: + + $ cd ptxdist-|\verbatimcmd\ptxdistVendorVersion| + $ ls -lF + total 530 + -rw-r--r-- 1 jb user 18446 Sep 9 15:59 COPYING + -rw-r--r-- 1 jb user 4048 Sep 9 15:59 CREDITS + -rw-r--r-- 1 jb user 115540 Sep 9 15:59 ChangeLog + -rw-r--r-- 1 jb user 57 Sep 9 15:59 INSTALL + -rw-r--r-- 1 jb user 3868 Sep 9 15:59 Makefile.in + -rw-r--r-- 1 jb user 4268 Sep 9 15:59 README + -rw-r--r-- 1 jb user 2324 Sep 9 15:59 README.devel + -rw-r--r-- 1 jb user 63516 Sep 9 15:59 TODO + -rwxr-xr-x 1 jb user 28 Sep 9 15:59 autogen.sh + drwxr-xr-x 2 jb user 72 Sep 9 15:59 bin + drwxr-xr-x 12 jb user 352 Sep 9 15:59 config + -rwxr-xr-x 1 jb user 224087 Sep 9 17:34 configure + -rw-r--r-- 1 jb user 12196 Sep 9 15:59 configure.ac + drwxr-xr-x 10 jb user 248 Sep 9 15:59 generic + drwxr-xr-x 242 jb user 8168 Sep 9 15:59 patches + drwxr-xr-x 2 jb user 1624 Sep 9 15:59 platforms + drwxr-xr-x 4 jb user 112 Sep 9 15:59 plugins + lrwxrwxrwx 1 jb user 7 Oct 4 20:42 projectroot -> generic + drwxr-xr-x 6 jb user 60664 Sep 9 15:59 rules + drwxr-xr-x 9 jb user 936 Sep 9 15:59 scripts + drwxr-xr-x 2 jb user 512 Sep 9 15:59 tests + +Prerequisites +~~~~~~~~~~~~~ + +Before PTXdist can be installed it has to be checked if all necessary +programs are installed on the development host. The configure script +will stop if it discovers that something is missing. + +The PTXdist installation is based on GNU autotools, so the first thing +to be done now is to configure the packet: + +:: + + $ ./configure + +This will check your system for required components PTXdist relies on. +If all required components are found the output ends with: + +:: + + [escapechar=^] + [...] + checking whether python development files are present... yes + checking for patch... /usr/bin/patch + checking whether /usr/bin/patch will work... yes + + configure: creating ./config.status + config.status: creating Makefile + + ptxdist version \ptxdistVendorVersion configured. + Using '/usr/local' for installation prefix. + + Report bugs to ptxdist@pengutronix.de + +Without further arguments PTXdist is configured to be installed into +``/usr/local``, which is the standard location for user installed +programs. To change the installation path to anything non-standard, we +use the ``--prefix`` argument to the ``configure`` script. The +``--help`` option offers more information about what else can be changed +for the installation process. + +The installation paths are configured in a way that several PTXdist +versions can be installed in parallel. So if an old version of PTXdist +is already installed there is no need to remove it. + +One of the most important tasks for the ``configure`` script is to find +out if all the programs PTXdist depends on are already present on the +development host. The script will stop with an error message in case +something is missing. If this happens, the missing tools have to be +installed from the distribution befor re-running the ``configure`` +script. + +When the ``configure`` script is finished successfully, we can now run + +:: + + $ make + +All program parts are being compiled, and if there are no errors we can +now install PTXdist into it’s final location. In order to write to +``/usr/local``, this step has to be performed as user *root*: + +:: + + $ sudo make install + [enter password] + [...] + +If we don’t have root access to the machine it is also possible to +install PTXdist into some other directory with the ``--prefix`` option. +We need to take care that the ``bin/`` directory below the new +installation dir is added to our ``$PATH`` environment variable (for +example by exporting it in ``~/.bashrc``). + +The installation is now done, so the temporary folder may now be +removed: + +:: + + $ cd ../../ + $ rm -fr local + +Configuring PTXdist +~~~~~~~~~~~~~~~~~~~ + +When using PTXdist for the first time, some setup properties have to be +configured. Two settings are the most important ones: where to store the +source archives and if a proxy must be used to gain access to the world +wide web. + +Run PTXdist’s setup: + +:: + + $ ptxdist setup + +Due to the fact that PTXdist is working with sources only, it needs +various source archives from the world wide web. If these archives are +not present on our host, PTXdist starts the ``wget`` command to download +them on demand. + +Proxy Setup +^^^^^^^^^^^ + +To do so, an internet access is required. If this access is managed by a +proxy ``wget`` command must be advised to use it. PTXdist can be +configured to advise the ``wget`` command automatically: navigate to +entry *Proxies* and enter the required addresses and ports to access the +proxy in the form: + +``://
:`` + +Source Archive Location +^^^^^^^^^^^^^^^^^^^^^^^ + +Whenever PTXdist downloads source archives it stores these archives in a +project local manner. This is the default behaviour. If we are working +with more than one PTXdist based project, every project would download +its own required archives in this case. To share all source archives +between all projects, PTXdist can be configured to share only one +archive directory for all projects it handles: navigate to menu entry +*Source Directory* and enter the path to the directory where PTXdist +should store archives to share between its projects. + +Generic Project Location +^^^^^^^^^^^^^^^^^^^^^^^^ + +If we already installed the generic projects we should also configure +PTXdist to know this location. If we already did so, we can use the +command ``ptxdist projects`` to get a list of available projects and +``ptxdist clone`` to get a local working copy of a shared generic +project. + +Navigate to menu entry *Project Searchpath* and enter the path to +projects that can be used in such a way. Here we can configure more than +one path, each part can be delemited by a colon. For example for +PTXdist’s generic projects and our own previous projects like this: + +``/usr/local/lib/ptxdist-/projects:/office/my_projects/ptxdist`` + +Leave the menu and store the configuration. PTXdist is now ready for +use. + +If there is no toolchain available yet, the next step is to build one at +least for the desired target architecture. Refer to the application note +“Building Toolchains” for further details. + +In order to build the toolchain in the next step, the specific PTXdist- +is required. We must repeat the previous steps with the PTXdist- to +install it on our host as well. All PTXdist revisions can co-exist. + +Toolchains +---------- + +Before we can start building our first userland we need a cross +toolchain. On Linux, toolchains are no monolithic beasts. Most parts of +what we need to cross compile code for the embedded target comes from +the *GNU Compiler Collection*, ``gcc``. The gcc packet includes the +compiler frontend, ``gcc``, plus several backend tools (cc1, g++, ld +etc.) which actually perform the different stages of the compile +process. ``gcc`` does not contain the assembler, so we also need the +*GNU Binutils package* which provides lowlevel stuff. + +Cross compilers and tools are usually named like the corresponding host +tool, but with a prefix – the *GNU target*. For example, the cross +compilers for ARM and powerpc may look like + +``arm-softfloat-linux-gnu-gcc`` + +``powerpc-unknown-linux-gnu-gcc`` + +With these compiler frontends we can convert e.g. a C program into +binary code for specific machines. So for example if a C program is to +be compiled natively, it works like this: + +:: + + $ gcc test.c -o test + +To build the same binary for the ARM architecture we have to use the +cross compiler instead of the native one: + +:: + + $ arm-softfloat-linux-gnu-gcc test.c -o test + +Also part of what we consider to be the “toolchain” is the runtime +library (libc, dynamic linker). All programs running on the embedded +system are linked against the libc, which also offers the interface from +user space functions to the kernel. + +The compiler and libc are very tightly coupled components: the second +stage compiler, which is used to build normal user space code, is being +built against the libc itself. For example, if the target does not +contain a hardware floating point unit, but the toolchain generates +floating point code, it will fail. This is also the case when the +toolchain builds code for i686 CPUs, whereas the target is i586. + +So in order to make things working consistently it is necessary that the +runtime libc is identical with the libc the compiler was built against. + +PTXdist doesn’t contain a pre-built binary toolchain. Remember that it’s +not a distribution but a development tool. But it can be used to build a +toolchain for our target. Building the toolchain usually has only to be +done once. It may be a good idea to do that over night, because it may +take several hours, depending on the target architecture and development +host power. + +Using existing Toolchains from different Vendors +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If a toolchain from a different vendor than OSELAS is already installed +which is known to be working, the toolchain building step with PTXdist +may be omitted. + +The OSELAS.BoardSupport() Packages shipped for PTXdist have been tested +with the OSELAS.Toolchains() built with the same PTXdist version. So if +an external toolchain is being used which isn’t known to be stable, a +target may fail. Note that not all compiler versions and combinations +work properly in a cross environment. + +Every OSELAS.BoardSupport() Package checks for its OSELAS.Toolchain it’s +tested against, so using a toolchain from a different vendor than OSELAS +requires an additional step: + +Open the OSELAS.BoardSupport() Package menu with: + +:: + + $ ptxdist platformconfig + +and navigate to ``architecture ---> toolchain`` and +``check for specific toolchain vendor``. Clear this entry to disable the +toolchain vendor check. + +Preconditions a toolchain from a different vendor than OSELAS must meet: + +- it shall be built with the configure option ``--with-sysroot`` + pointing to its own C libraries. + +- it should not support the *multilib* feature as this may confuse + PTXdist which libraries are to select for the root filesystem + +If we want to check if our toolchain was built with the +``--with-sysroot`` option, we just run this simple command: + +:: + + $ mytoolchain-gcc -v 2|\textgreater{}\&|1 | grep with-sysroot + +If this command **does not** output anything, this toolchain was not +built with the ``--with-sysroot`` option and cannot be used with +PTXdist. + +Omitting building a Toolchain +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Pengutronix also provides ’ready to use’ toolchains in a binary manner. +These toolchains are built from the OSELAS.Toolchain bundle, so they +comply with all of Pengutronix’s board support packages and we can use +them instead of building our own one. + +The binary OSELAS toolchains are provided as *Debian Distribution +Packages*. Also most non-Debian distributions can handle such packages. + +In order to install the OSELAS binary toolchains on a Debian based +system the following steps are required: + +Add the OSELAS Server as a Package Source +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To register the OSELAS package server to the list of known package +servers, we add a new file with the name ``pengutronix.list`` into the +directory ``/etc/apt/sources.list.d/``. The basename of this file isn’t +important, while the extension ``.list`` is. + +The contents of this new file describe the Pengutronix server as an +available package source. It is defined via one text line: + +:: + + deb http://debian.pengutronix.de/debian/ sid main contrib non-free + +Note: if the directory ``/etc/apt/sources.list.d/`` does not exist, the +text line mentioned above must be added to the file +``/etc/apt/sources.list`` instead. + +Make the OSELAS Server Content available +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The package manager now must update its packages list with the following +command: + +:: + + $ apt-get update + +Install the Archive Keyring +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To avoid warnings about untrusted package sources we can install the +OSELAS archive keyring with the following command: + +:: + + $ apt-get install pengutronix-archive-keyring + +Install the binary OSELAS Toolchain +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Now everything is in place to install the binary OSELAS toolchain for +the board support package: + +:: + + $ apt-get install oselas.toolchain-|\oselasTCNVendorVersion \oselasTCNVendorPatchLevel -\ptxdistCompilerName-\ptxdistCompilerVersion| + +These package names are very long and hard to type without making typos. +An easier way is to ask the package manager for available toolchains and +just use the name by copy and paste it. + +:: + + $ apt-cache search "oselas.toolchain-.*-|\oselasTCNarch|.*|\oselasTCNvariant|.*" + oselas.toolchain-|\oselasTCNVendorVersion \oselasTCNVendorPatchLevel -\ptxdistCompilerName-\ptxdistCompilerVersion| + +The binary OSELAS Toolchain Package for non-Debian Distributions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The *Debian Distribution Packages* can be found on our server at +http://debian.pengutronix.de/debian/pool/main/o/ + +The related OSELAS toolchain package can be found here: + +| Subpath is: +| ``oselas.toolchain---/`` +| Package filename is: +| ``oselas.toolchain---.deb``. + +Package filenames for 32 bit host machines are ending on ``*_i386.deb`` +and for 64 bit host machines on ``*_amd64.deb``. + +Building a Toolchain +~~~~~~~~~~~~~~~~~~~~ + +PTXdist handles toolchain building as a simple project, like all other +projects, too. So we can download the OSELAS.Toolchain bundle and build +the required toolchain for the OSELAS.BoardSupport() Package. + +Building any toolchain of the OSELAS.Toolchain- is tested with PTXdist-. +Pengutronix recommends to use this specific PTXdist to build the +toolchain. So, it might be essential to install more than one PTXdist +revision to build the toolchain and later on the Board Support Package +if the latter one is made for a different PTXdist revision. + +A PTXdist project generally allows to build into some project defined +directory; all OSELAS.Toolchain projects that come with PTXdist are +configured to use the standard installation paths mentioned below. + +All OSELAS.Toolchain projects install their result into +``/opt/OSELAS.Toolchain-/``. + +| Usually the ``/opt`` directory is not world writeable. So in order to + build our OSELAS.Toolchain into that directory we need to use a root + account to change the permissions. PTXdist detects this case and asks + if we want to run ``sudo`` to do the job for us. Alternatively we can + enter: +| ``mkdir /opt/OSELAS.Toolchain- chown /opt/OSELAS.Toolchain- chmod a+rwx /opt/OSELAS.Toolchain-``. + +We recommend to keep this installation path as PTXdist expects the +toolchains at ``/opt``. Whenever we go to select a platform in a +project, PTXdist tries to find the right toolchain from data read from +the platform configuration settings and a toolchain at ``/opt`` that +matches to these settings. But that’s for our convenience only. If we +decide to install the toolchains at a different location, we still can +use the *toolchain* parameter to define the toolchain to be used on a +per project base. + +Building the OSELAS.Toolchain for +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Do the following steps in your own home directory ($HOME). The final +OSELAS.Toolchain gets installed to ``opt/``, but must **never** be +compiled in the **opt/** directory. You will get many funny error +messages, if you try to compile the OSELAS-Toolchain in **opt/**. + +To compile and install an OSELAS.Toolchain we have to extract the +OSELAS.Toolchain archive, change into the new folder, configure the +compiler in question and start the build. + +The required compiler to build the board support package is + +```` + +So the steps to build this toolchain are: + +Naming Conventions used to describe a Toolchain +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The OSELAS.Toolchain archive comes with various predefined toolchains. +To select the correct one for our project we can follow the project +specific documentation or, if we know what revision composition we need, +we can find the correct configuration file in the following way. + +| For example this is one of the predefined toolchains: + +**i586-unknown-linux-gnu\_gcc-4.7.2\_glibc-2.16.0\_binutils-2.22\_kernel-3.6-sanitized** + +The first part (i586-unknown-linux-gnu) defines the target architecture +and the target system. In this example it’s an ia32 architecture based +system, with at least i586 classes (or higher) of processors. + +The next part (gcc-4.7.2) defines the compiler release, what revision of +the basic C library is used (glibc-2.16.0) and what release of binutils +(binutils-2.22) is used in this toolchain. + +The last part (kernel-3.6) defines the kernel header in use for this +toolchain. This is important, as applications built with this toolchain +must be run on **at least** this kernel revision. + +We can find all predefined toolchains inside the ``ptxconfigs/`` +directory and its subdirectories. + +Building a Sample OSELAS.Toolchain +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To compile and install an OSELAS.Toolchain we have to extract the +OSELAS.Toolchain archive, change into the new folder, configure the +compiler in question and start the build. + +As an example we will build the + +```` + +toolchain. + +The steps to do so are: + +In order to build any of the OSELAS.Toolchains, the host must provide +the tool *fakeroot*. Otherwise the +message\ ``bash: fakeroot: command not found`` will occur and the build +stops. + +Please ensure the ’current directory’ (the ``.`` entry) is not part of +your PATH environment variable. PTXdist tries to sort out this entry, +but might not be successful in doing so. Check by running +``ptxdist print PATH`` if the output still contains any kind of ’current +directory’ as a component. If yes, remove it first. + +:: + + $ tar xf OSELAS.Toolchain-|\verbatimcmd\oselasTCNVendorVersion{}\oselasTCNVendorPatchLevel|.tar.bz2 + $ cd OSELAS.Toolchain-|\verbatimcmd\oselasTCNVendorVersion{}\oselasTCNVendorPatchLevel| + $ ^ptxdist-|\oselasTCNVendorptxdistversion{}| select ptxconfigs/|\verbatimcmd\ptxcr| + |\verbatimcmd\oselasToolchainName|.ptxconfig^ + $ ptxdist-|\oselasTCNVendorptxdistversion{}| go + +At this stage we have to go to our boss and tell him that it’s probably +time to go home for the day. Even on reasonably fast machines the time +to build an OSELAS.Toolchain is something like around 30 minutes up to a +few hours. + +Measured times on different machines: + +- Single Pentium 2.5 GHz, 2 GiB RAM: about 2 hours + +- Turion ML-34, 2 GiB RAM: about 1 hour 30 minutes + +- Dual Athlon 2.1 GHz, 2 GiB RAM: about 1 hour 20 minutes + +- Dual Quad-Core-Pentium 1.8 GHz, 8 GiB RAM: about 25 minutes + +- 24 Xeon cores 2.54 GHz, 96 GiB RAM: about 22 minutes + +Another possibility is to read the next chapters of this manual, to find +out how to start a new project. + +When the OSELAS.Toolchain project build is finished, PTXdist is ready +for prime time and we can continue with our first project. + +Protecting the Toolchain +~~~~~~~~~~~~~~~~~~~~~~~~ + +All toolchain components are built with regular user permissions. In +order to avoid accidential changes in the toolchain, the files should be +set to read-only permissions after the installation has finished +successfully. It is also possible to set the file ownership to root. +This is an important step for reliability, so it is highly recommended. + +Building additional Toolchains +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The OSELAS.Toolchain- bundle comes with various predefined toolchains. +Refer the ``ptxconfigs/`` folder for other definitions. To build +additional toolchains we only have to clean our current toolchain +project, removing the current ``selected_ptxconfig`` link and creating a +new one. + +:: + + $ ptxdist clean + $ rm selected_ptxconfig + $ ptxdist select ptxconfigs/any_other_toolchain_def.ptxconfig + $ ptxdist go + +All toolchains will be installed side by side architecture dependent +into directory + +``/opt/OSELAS.Toolchain-/architecture_part``. + +Different toolchains for the same architecture will be installed side by +side version dependent into directory + +``/opt/OSELAS.Toolchain-/architecture_part/version_part``. + diff --git a/doc/getting_help.rst b/doc/getting_help.rst new file mode 100644 index 000000000..d49ec6602 --- /dev/null +++ b/doc/getting_help.rst @@ -0,0 +1,84 @@ +Getting help +============ + +Below is a list of locations where you can get help in case of trouble. +For questions how to do something special with PTXdist or general +questions about Linux in the embedded world, try these. + +Mailing Lists +------------- + +About PTXdist in Particular +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is an English language public mailing list for questions about +PTXdist. See + +http://www.pengutronix.de/mailinglists/index_en.html + +on how to subscribe to this list. If you want to search through the +mailing list archive, visit + +http://www.mail-archive.com/ + +and search for the list *ptxdist*. Please note again that this mailing +list is just related to the PTXdist as a software. For questions +regarding your specific BSP, see the following items. + +About Embedded Linux in General +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is a German language public mailing list for general questions +about Linux in embedded environments. See + +http://www.pengutronix.de/mailinglists/index_de.html + +on how to subscribe to this list. Note: You can also send mails in +English. + +About Working on the Linux Kernel +--------------------------------- + +The book *Linux Kernel in a Nutshell* from *Greg Kroah-Hartman*. Its +online version can be read here: + +http://www.kroah.com/lkn/ + +Chat/IRC +-------- + +**About PTXdist in particular** + +**irc.freenode.net:6667** + +Create a connection to the **irc.freenode.net:6667** server and enter +the chatroom **#ptxdist**. This is an English room to answer questions +about PTXdist. Best time to meet somebody there is at European daytime. + +Commercial Support +------------------ + +You can order immediate support through customer specific mailing lists, +by telephone or also on site. Ask our sales representative for a price +quotation for your special requirements. + +Contact us at: + +**Pengutronix +Peiner Str. 6-8 +31137 Hildesheim +Germany +Phone: +49 - 51 21 / 20 69 17 - 0 +Fax: +49 - 51 21 / 20 69 17 - 55 55** + +or by electronic mail: + +sales@pengutronix.de + +.. |image| image:: figures/gup-self-documenting-code.jpg +.. |image| image:: figures/gup-turing-complete.jpg +.. |image| image:: figures/gup-project-folder.jpg +.. |image| image:: figures/gup-never-ever.jpg +.. |image| image:: figures/gup-hope-and-pray.jpg +.. |image| image:: figures/gup-git-blame2.jpg + diff --git a/doc/index.rst b/doc/index.rst index e70e9fbf9..bf27f63c5 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -13,7 +13,15 @@ Contents: :numbered: :maxdepth: 1 - macro_reference + welcome + environment + user_manual + dev_manual + ref_manual + daily_work + revisions + thanks + getting_help * :ref:`search` * :ref:`genindex` diff --git a/doc/ref_manual.rst b/doc/ref_manual.rst new file mode 100644 index 000000000..1221166df --- /dev/null +++ b/doc/ref_manual.rst @@ -0,0 +1,1076 @@ +PTXdist Reference +================= + +|image| + +Variables Reference +------------------- + +The following variables are provided by PTXdist to simplify creating +rule files. Every developer should use these variables in every single +line in the rule file to avoid any further adaption when external paths +are changed. + +To get their content related to the current project, we can simply run +a: + +:: + + @\$@ ptxdist print PTXDIST_TOPDIR + /usr/local/lib/ptxdist-@\releasenumber{}@ + +Replace the ``PTXDIST_TOPDIR`` with one of the other generic variables +PTXdist provides. + +PTXDIST\_TOPDIR +~~~~~~~~~~~~~~~ + +Points always to the installation directory of PTXdist. + +PTXDIST\_WORKSPACE +~~~~~~~~~~~~~~~~~~ + +Everything that references ``PTXDIST_WORKSPACE`` will use the active +projects’s folder. + +PTXDIST\_SYSROOT\_CROSS +~~~~~~~~~~~~~~~~~~~~~~~ + +``PTXDIST_SYSROOT_CROSS`` points to a directory tree all cross relevant +executables, libraries and header files are installed to in the current +project. All of the project’s packages built for the host to create data +for the target are searching in this directory tree for their +dependencies (executables, header and library files). Use +``$(PTXDIST_SYSROOT_CROSS)/bin`` to install executables, +``$(PTXDIST_SYSROOT_CROSS)/include`` for header files and +``$(PTXDIST_SYSROOT_CROSS)/lib`` for libraries. + +PTXDIST\_SYSROOT\_HOST +~~~~~~~~~~~~~~~~~~~~~~ + +``PTXDIST_SYSROOT_HOST`` points to a directory tree all host relevant +executables, libraries and header files are installed to. All project’s +packages built for the host are searching in this directory tree for +their dependencies (executables, header and library files). Use +``$(PTXDIST_SYSROOT_HOST)/bin`` to install executables, +``$(PTXDIST_SYSROOT_HOST)/include`` for header files and +``$(PTXDIST_SYSROOT_HOST)/lib`` for libraries. + +PTXDIST\_SYSROOT\_TARGET +~~~~~~~~~~~~~~~~~~~~~~~~ + +``PTXDIST_SYSROOT_TARGET`` points to a directory tree all target +relevant libraries and header files are installed to. All project’s +packages built for the target are searching in this directory tree for +their dependencies (header and library files). These files are for +compile time only (for example to link a target executable against a +target library), not for runtime! Use +``$(PTXDIST_SYSROOT_TARGET)/include`` for header files and +``$(PTXDIST_SYSROOT_TARGET)/lib`` for libraries. + +Other useful variables: + +CROSS\_PATH +~~~~~~~~~~~ + +Use to find cross tools. This path must be used to create anything that +depends on the target’s architecture, but needs something running on the +host to do the job. Examples: + +Creating a jffs2 image from the target’s root filesystem + This will need a tool running on the host, but it will create data + or code that runs on or is used on the target + +Building a library for the target + If this library needs other resources to be built (other libraries) + its ``configure`` finds the right information in this path. + +HOST\_PATH +~~~~~~~~~~ + +Used to find host tools. This path must be used to create anything that +not depends on the architecture. + +ROOTDIR +~~~~~~~ + +``ROOTDIR`` points to the root of the target’s root filesystem in the +current project. Used in very rare cases (to create strange packages +based on data in target’s root filesystem for example). + +PTXCONF\_PLATFORM +~~~~~~~~~~~~~~~~~ + +``PTXCONF_PLATFORM`` expands to the name of the currently selected +platform. This name is used in various file names and paths. + +PTXDIST\_PLATFORMSUFFIX +~~~~~~~~~~~~~~~~~~~~~~~ + +``PTXDIST_PLATFORMSUFFIX`` expands to the name of the currently selected +platform, but with a leading dot. This is used in various files PTXdist +should search for. + +PTXDIST\_PLATFORMCONFIGDIR +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``PTXDIST_PLATFORMCONFIGDIR`` points to the directory tree of the +currently selected platform. This path is used in various search +functions. + +PTXDIST\_PLATFORMDIR +~~~~~~~~~~~~~~~~~~~~ + +``PTXDIST_PLATFORMDIR`` points to the directory build tree of the +currently selected platform. + +Rule File Macro Reference +------------------------- + +Rules files in PTXdist are using macros to get things work. Its highly +recommended to use these macros instead to do something by ourself. +Using the macros is portable and such easier to maintain in the case a +project should be upgraded to a more recent PTXdist version. + +This chapter describes the predefined macros in PTXdist and their usage. + +Whenever one of the macros installs something to the target’s root +filesystem, it also accepts user and group IDs which are common in all +filesysems Linux supports. These IDs can be given as numerical values +and as text strings. In the case text strings are given PTXdist converts +them into the coresponding numerical value based on the BSP local file +FIXME “projectroot/etc/” + +Note: This list is not complete yet. + +targetinfo +~~~~~~~~~~ + +Usage: + +:: + + $(call targetinfo) + +Gives a feedback, what build *stage* is just started. Thats why it +should always be the first call for each *stage*. For the package *foo* +and the *compile stage* it will output: + +:: + + -------------------- + target: foo.compile + -------------------- + +touch +~~~~~ + +Usage: + +:: + + $(call touch) + +Gives a feedback, what build *stage* is just finished. Thats why it +should always be the last call for each *stage*. For the package *foo* +and the *compile stage* it will output: + +:: + + finished target foo.compile + +clean +~~~~~ + +Usage: + +:: + + $(call clean, ) + +Removes the given directory ````. + +install\_copy +~~~~~~~~~~~~~ + +Usage: + +:: + + $(call install_copy, , , , , [, [, ]]) + +Installs given file or directory into: + +- the project’s ``root/`` + +- the project’s ``root-debug/`` + +- an ipkg/opkg packet in the project’s ``packages/`` + +Some of the parameters have fixed meanings: + + + Name of the IPKG/OPKG the macro should work on + + + User ID the file should use in the target’s root filesystem + + + Group ID the file should use in the target’s root filesystem + + + Permission (in an octal value) the file should use in the target’s + root filesystem + +The remaining parameters vary with the use case: + +The parameter can be: + +- a directory path that should be created in the target’s root + filesystem. In this case the must be omitted. The given + path must always start with a ``/`` and means the root of the + target’s filesystem. + +- an absolute path to a file that should be copied to the target’s root + filesystem. To avoid fixed paths, all packages are providing the + \_DIR variable. So, this parameter in our *foo* example + package can be a ``$(FOO_DIR)/foo``. + +- a minus sign (``-``). PTXdist uses the parameter in + this case to locate the file to copy from. This only works if the + package uses the default *install* stage. Only in this case an + additional folder in ``packages`` will be created for the package and + its files. For our *foo* example package this directory is + ``packages/foo-1.1.0``. + +The parameter can be: + +- omitted if a directory in target’s root filesystem should be created. + For this case the directory to be created is in the + parameter. + +- | an absolute path and filename with its root in target’s root + filesysem. It must start with a slash (``/``). If also the + parameter was given, the file can be renamed while copying. + | If the parameter was given as a minus sign (``-``) the + is also used to locate the source. For our *foo* + example package if we give as ``/usr/bin/foo``, + PTXdist copies the file ``packages/foo-1.1.0/usr/bin/foo`` + +The is a complete optional parameter to prevent this macro from +the regular stripping process it does on files. Most of the cases +stripping debug information from files is intended. But some kind of +files getting destroyed when this stripping happens to them. One example +is a Linux kernel module. If it gets stripped, it can’t be loaded into +the kernel anymore. + +full strip + fully strip the file while installing when this parameter is **y** + or not given at all (default case). + +partially strip + only strips real debug information from the file when this parameter + is **k**. Useful to keep Linux kernel module loadable at run-time + +no strip + preserve the file from being stripped when this parameter is one of + the following: **0**, **n**, **no**, **N** or **NO**. + +Due to the complexity of this macro, here are some usage examples: + +Create a directory in the root filesystem: + +:: + + $(call install_copy, foo, 0, 0, 0755, /home/user-foo) + +Copy a file from the package build directory to the root filesystem: + +:: + + $(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/foo) + +Copy a file from the package build directory to the root filesystem and +rename it: + +:: + + $(call install_copy, foo, 0, 0, 0755, $(FOO_DIR)/foo, /usr/bin/bar) + +Copy a file from the package install directory to the root filesystem: + +:: + + $(call install_copy, foo, 0, 0, 0755, -, /usr/bin/foo) + +install\_tree +~~~~~~~~~~~~~ + +Usage: + +:: + + $(call install_tree, , , , , ) + +Installs the whole directory tree with all files from the given +directory into: + +- the project’s ``root/`` + +- the project’s ``root-debug/`` + +- an ipkg packet in the project’s ``packages/`` + +Some of the parameters have fixed meanings: + + + Name of the IPKG/OPKG the macro should work on + + + User ID the directories and files should use in the target’s root + filesystem or ``-`` to keep the UID from the source tree + + + Group ID the directories and files should use in the target’s root + filesystem or ``-`` to keep the GID from the source tree + + + This is the path to the tree of directories and files to be + installed. It can be ``-`` to use the package directory of the + current package instead + + + The basename of the to-be-installed tree in the root filesystem + +Note: This installation macro + +- uses the same permission flags in the destination dir as found in the + source dir. This is valid for directories and regular files + +- skips all directories with names like ``.svn``, ``.git``, ``.pc`` and + ``CVS`` in the source directory + +Examples: + +Install the whole tree found in ``/home/jbe/foo`` to the root filesystem +at location ``/usr/share/bar``. + +:: + + $(call install_tree, foo, 0, 0, /home/jbe/foo, /usr/share/bar) + +Install all files from the tree found in the current package FOO to the +root filesystem at location ``/usr/share/bar``. + +:: + + $(call install_tree, foo, 0, 0, -, /usr/share/bar) + +If the current package is ``foo-1.0`` the base path for the directory +tree will be ``$(PKGDIR)/foo-1.0/usr/share/bar``. + +install\_alternative\_tree +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Usage: + +:: + + $(call install_alternative_tree, , , , ) + +Installs the whole source directory tree with all files from the given +directory into: + +- the project’s ``root/`` + +- the project’s ``root-debug/`` + +- an ipkg packet in the project’s ``packages/`` + +The is used like in the ``install_alternative`` to let +PTXdist search in the same directories and order for the given +directory. + +Some of the parameters have fixed meanings: + + + Name of the IPKG/OPKG the macro should work on + + + User ID the directories and files should use in the target’s root + filesystem or ``-`` to keep the UID from the source + + + Group ID the directories and files should use in the target’s root + filesystem or ``-`` to keep the GID from the source + + + The basename of the to-be-installed tree in the root filesystem + +Note: This installation macro + +- uses the same permission flags in the destination dir as found in the + source dir. This is valid for directories and regular files + +- skips all directories with names like ``.svn``, ``.git``, ``.pc`` and + ``CVS`` in the source directory + +Examples: + +Install the whole tree found in project’s ``projectroot/usr/share/bar`` +to the root filesystem at location ``/usr/share/bar``. + +:: + + $(call install_alternative_tree, foo, 0, 0, /usr/share/bar) + +install\_alternative +~~~~~~~~~~~~~~~~~~~~ + +Usage: + +:: + + $(call install_alternative, , , , , ) + +Installs given files or directories into: + +- the project’s ``root/`` + +- the project’s ``root-debug/`` + +- an ipkg/opkg packet in the project’s ``packages/`` + +The base parameters and their meanings: + + + Name of the IPKG/OPKG the macro should work on + + + User ID the file should use in the target’s root filesystem + + + Group ID the file should use in the target’s root filesystem + + + Permission (in an octal value) the file should use in the target’s + root filesystem + +The parameter is meant as an absolute path and filename in +target’s root filesystem. PTXdist searches for the source of this file +in: + +- the local project + +- in the used platform + +- PTXdist’s install path + +- in the current package + +As this search algorithm is complex, here an example for the file +``/etc/foo`` in package ``FOO``. PTXdist will search for this file in +the following order: + +- project’s directory ``projectroot./etc/foo`` + +- project’s directory ``projectroot/etc/foo.`` + +- platform’s directory ``configs//projectroot/etc/foo`` + +- project’s directory ``projectroot/etc/foo`` + +- ptxdist’s directory ``generic/etc/foo`` + +- project’s directory ``$(FOO_DIR)/etc/foo`` + +The generic rules are looking like the following: + +- ``$(PTXDIST_WORKSPACE)/projectroot$(PTXDIST_PLATFORMSUFFIX)/etc/foo`` + +- ``$(PTXDIST_WORKSPACE)/projectroot/etc/foo$(PTXDIST_PLATFORMSUFFIX)`` + +- ``$(PTXDIST_PLATFORMCONFIGDIR)/projectroot/etc/foo`` + +- ``$(PTXDIST_WORKSPACE)/projectroot/etc/foo`` + +- ``$(PTXDIST_TOPDIR)/generic/etc/foo`` + +- ``$(FOO_DIR)/etc/foo`` + +Note: You can get the current values for the listed variables above via +running PTXdist with the ``print`` parameter: + +:: + + ptxdist print PTXDIST_PLATFORMSUFFIX + +install\_link +~~~~~~~~~~~~~ + +Usage: + +:: + + $(call install_link, , , ) + +Installs a symbolic link into: + +- the project’s ``root/`` + +- the project’s ``root-debug/`` + +- an ipkg/opkg packet in the project’s ``packages/`` + +The parameters and their meanings: + + + Name of the IPKG/OPKG the macro should work on + + + Path and name the link should point to. Note: This macro rejects + absolute paths. If needed use relative paths instead. + + + Path and name of the symbolic link. + +A few usage examples. + +Create a symbolic link as ``/usr/lib/libfoo.so`` pointing to +``libfoo.so.1.1.0`` in the same directory: + +:: + + $(call install_link, foo, libfoo.so.1.1.0, /usr/lib/libfoo.so) + +Create a symbolic link as ``/usr/bin/foo`` pointing to ``/bin/bar``: + +:: + + $(call install_link, foo, ../../bin/bar, /usr/bin/foo) + +install\_archive +~~~~~~~~~~~~~~~~ + +Usage: + +:: + + $(call install_archive, , , , , ) + +Installs archives content into: + +- the project’s ``root/`` + +- the project’s ``root-debug/`` + +- an ipkg/opkg packet in the project’s ``packages/`` + +All parameters have fixed meanings: + + + Name of the IPKG/OPKG the macro should work on + + + User ID all files and directory of the archive should use in the + target’s root filesystem. A ``-`` uses the file’s/directory’s UID in + the archive + + + Group ID the files and directories should use in the target’s root + filesystem. A ``-`` uses the file’s/directory’s GID in the archive + + + Name of the archive to be used in this call. The given path and + filename is used as is + + + Base path component in the root filesystem the archive should be + extracted to. Can be just ``/`` for root. + +install\_lib +~~~~~~~~~~~~ + +Usage: + +:: + + $(call install_lib, , , , , ) + +Installs the shared library into the root filesystem. + +- the project’s ``root/`` + +- the project’s ``root-debug/`` + +- an ipkg/opkg packet in the project’s ``packages/`` + +The parameters and their meanings: + + + Name of the IPKG/OPKG the macro should work on + + + User ID the file should use in the target’s root filesystem + + + Group ID the directories and files should use in the target’s root + filesystem + + + Permission (as an octal value) the library should use in the + target’s root filesystem (mostly 0644) + + + Basename of the library without any extension and path + +The ``install_lib`` macro searches for the library at the most common +directories ``/lib`` and ``/usr/lib``. And it searches always in the +package’s corresponding directory in ``packages/``. It also handles all +required links to make the library work at runtime. + +An example. + +Lets assume the package ’foo-1.0.0’ has installed the library ``libfoo`` +into its ``packages/foo-1.0.0`` at: + +- the lib: ``packages/foo-1.0.0/usr/lib/libfoo1.so.0.0.0`` + +- first link: ``packages/foo-1.0.0/usr/lib/libfoo1.so.0`` + +- second link: packages/foo-1.0.0/usr/lib/libfoo1.so + +To install this library and its corresponding links, the following line +does the job: + +:: + + $(call install_lib, foo, 0, 0, 0644, libfoo1) + +Note: The package’s install stage must be ’DESTDIR’ aware to be able to +make it install its content into the corresponding packages directory +(in our example ``packages/foo-1.0.0/`` here). + +ptx/endis +~~~~~~~~~ + +[ref:sub:`p`\ aram\ :sub:`e`\ ndis] + +To convert the state (set/unset) of a variable into an +``enable/disable`` string use the ``ptx/endis`` macro. If the given + is set this macro expands to the string ``enable``, if unset +to ``disable`` instead. + +Usage: + +:: + + --$(call ptx/endis, )- + +An example: + +:: + + FOO_CONF_OPT += --$(call ptx/endis,FOO_VARIABLE)-something + +Depending on the state of FOO\_VARIABLE this line results into + +:: + + FOO_CONF_OPT += --enable-something (if FOO_VARIABLE is set) + FOO_CONF_OPT += --disable-something (if FOO_VARIABLE is unset) + +Refer ``ptx/disen`` for the opposite string expansion. + +ptx/disen +~~~~~~~~~ + +To convert the state (set/unset) of a variable into a ``disable/enable`` +string use the ``ptx/disen`` macro. If the given is set this +macro expands to the string ``disable``, if unset to ``enable`` instead. + +Usage: + +:: + + --$(call ptx/disen, )- + +An example: + +:: + + FOO_CONF_OPT += --$(call ptx/disen,FOO_VARIABLE)-something + +Depending on the state of FOO\_VARIABLE this line results into + +:: + + FOO_CONF_OPT += --disable-something (if FOO_VARIABLE is set) + FOO_CONF_OPT += --enable-something (if FOO_VARIABLE is unset) + +Refer ``ptx/endis`` for the opposite string expansion. + +ptx/wwo +~~~~~~~ + +To convert the state (set/unset) of a variable into a ``with/without`` +string use the ``ptx/wwo`` macro. If the given is set this +macro expands to the string ``with``, if unset to ``without`` instead. + +Usage: + +:: + + --$(call ptx/wwo, )- + +An example: + +:: + + FOO_CONF_OPT += --$(call ptx/wwo,FOO_VARIABLE)-something + +Depending on the state of FOO\_VARIABLE this line results into + +:: + + FOO_CONF_OPT += --with-something (if FOO_VARIABLE is set) + FOO_CONF_OPT += --without-something (if FOO_VARIABLE is unset) + +ptx/ifdef +~~~~~~~~~ + +To convert the state (set/unset) of a variable into one of two strings +use the ``ptx/ifdef`` macro. If the given is set this macro +expands to the first given string, if unset to the second given string. + +Usage: + +:: + + --with-something=$(call ptx/ifdef, , , : + @$(call targetinfo) + @$(call touch) + +Replace the by ``get``, ``extract``, ``prepare``, +``compile``, ``install`` or ``targetinstall``. + +PTXdist parameter reference +--------------------------- + +PTXdist is a command line tool, which is basicly called as: + +:: + + $ ptxdist [options] + +Setup and Project Actions +~~~~~~~~~~~~~~~~~~~~~~~~~ + +**``menu``**: this will start a menu front-end to control some of +PTXdist’s features in a menu based convenient way. This menu handles the +actions *menuconfig*, *platformconfig*, *kernel* config, *select*, +*platform*, *boardsetup*, *setup*, *go* and *images*. + +**``select ``**: this action will select a user land +configuration. This step is only required in projects, where no +``selected_ptxconfig`` file is present. The argument must point +to a valid user land configuration file. PTXdist provides this feature +to enable the user to maintain more than one user land configuration in +the same project. + +**``platform ``**: this action will select a platform +configuration. This step is only required in projects, where no +``selected_platform`` file is present. The argument must point +to a valid platform configuration file. PTXdist provides this feature to +enable the user to maintain more than one platform in one project. + +**``setup``**: PTXdist uses some global settings, independent from the +project it is working on. These settings belong to users preferences or +simply some network settings to permit PTXdist to download required +packages. + +**``boardsetup``**: PTXdist based projects can provide information to +setup and configure the target automatically. This action let the user +setup the environment specific settings like the network IP address and +so on. + +**``projects``**: if the generic projects coming in a separate archive +are installed, this actions lists the projects a user can clone for its +own work. + +**``clone ``**: this action clones an existing project from +the ``projects`` list into a new directory. The argument must be a +name gotten from ``ptxdist projects`` command, the argument is the +new project (and directory) name, created in the current directory. + +**``menuconfig``**: start the menu to configure the project’s root +filesystem. This is in respect to user land only. Its the main menu to +select applications and libraries, the root filesystem of the target +should consist of. + +**``menuconfig platform``**: this action starts the menu to configure +platform’s settings. As these are architecture and target specific +settings it configures the toolchain, the kernel and a bootloader (but +no user land components). Due to a project can support more than one +platform, this will configure the currently selected platform. The short +form for this action is ``platformconfig``. + +**``menuconfig kernel``**: start the menu to configure the platform’s +kernel. As a project can support more than one platform, this will +configure the currently selected platform. The short form for this +action is ``kernelconfig``. + +**``menuconfig barebox``**: this action starts the configure menu for +the selected bootloader. It depends on the platform settings which +bootloader is enabled and to be used as an argument to the +``menuconfig`` action parameter. Due to a project can support more than +one platform, this will configure the bootloader of the currently +selected platform. + +Build Actions +~~~~~~~~~~~~~ + +**``go``**: this action will build all enabled packages in the current +project configurations (platform and user land). It will also rebuild +reconfigured packages if any or build additional packages if they where +enabled meanwhile. If enables this step also builds the kernel and +bootloader image. + +**``images``**: most of the time this is the last step to get the +required files and/or images for the target. It creates filesystems or +device images to be used in conjunction with the target’s filesystem +media. The result can be found in the ``images/`` directory of the +project or the platform directory. + +Clean Actions +~~~~~~~~~~~~~ + +**``clean``**: the ``clean`` action will remove all generated files +while the last ``go`` run: all build, packages and root filesystem +directories. Only the selected configuration files are left untouched. +This is a way to start a fresh build cycle. + +**``clean root``**: this action will only clean the root filesystem +directories. All the build directories are left untouched. Using this +action will re-generate all ipkg/opkg archives from the already built +packages and also the root filesystem directories in the next ``go`` +action. The ``clean root`` and ``go`` action is useful, if the +*targetinstall* stage for all packages should run again. + +**``clean ``**: this action will only clean the dedicated +. It will remove its build directory and all installed files +from the corresponding sysroot directory. + +**``distclean``**: the ``distclean`` action will remove all files that +are not part of the main project. It removes all generated files and +directories like the ``clean`` action and also the created links in any +``platform`` and/or ``select`` action. + diff --git a/doc/revisions.rst b/doc/revisions.rst new file mode 100644 index 000000000..7f1dbcc32 --- /dev/null +++ b/doc/revisions.rst @@ -0,0 +1,57 @@ +Document Revisions +================== + ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/01/08 | Initial Revision | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/01/11 | Section “Adding simple Files to the Build Process” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/01/11 | Section “Variables reference” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/01/20 | Section “Debugging with CPU emulation” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/01/21 | Section “PTXdist parameter reference” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/04/15 | Section “Adding new Packages” extended | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/04/16 | Section “Using Existing Toolchains” extended | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/05/20 | Section “Adding binary only Files” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/05/20 | Section “Adding Files into the Build” foldet into “Adding binary only Files” | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2010/05/20 | Section “Adapting Userland Settings” extended | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2011/01/27 | Section “Rule File Macro Reference”, ’install\_tree’ documentation added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2011/01/27 | Section “Rule File Macro Reference”, ’install\_alternative’ search order fixed | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2011/01/27 | Section “Variables Reference”, some more useful variables added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2011/01/27 | Section “Adding new Packages”, documentation adapted to current ptxdist behaviour | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2011/10/21 | Section “Adding new Packages”, documentation adapted to current ptxdist behaviour | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2011/11/16 | Section “Using an External Kernel Source Tree”, documentation adapted to current ptxdist behaviour | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2012/04/04 | Section “Rule File Macro Reference”, add more common macros | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2012/04/05 | Section “Rule file layout”, documentation adapted to current ptxdist behaviour | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2013/11/13 | Section “Rule File Macro Reference”, documentation adapted to current ptxdist behaviour | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2013/12/09 | Section “Increasing Build Speed” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2013/12/12 | Comics of “geek & poke” added all over the place. Enjoy! | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2014/01/10 | Section “Using pre-build archives” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2014/06/19 | Section “Creating a new Autotools Based Library” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2014/07/07 | Section “Using a Distributed Compiler” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| 2014/07/07 | Section “Frequently Asked Questions (FAQ)” added | ++--------------+------------------------------------------------------------------------------------------------------+ +| | × | ++--------------+------------------------------------------------------------------------------------------------------+ + diff --git a/doc/thanks.rst b/doc/thanks.rst new file mode 100644 index 000000000..2284c40f9 --- /dev/null +++ b/doc/thanks.rst @@ -0,0 +1,14 @@ +Thanks +====== + +Usage of the “geek & poke” comics with kindly permission of Oliver +Widder: http://geek-and-poke.com + +|image| + +| The “geek & poke” comics are published under the Creative Commonse + License (CC BY 3.0). + +http://creativecommons.org/licenses/by/3.0/deed.en_US + + diff --git a/doc/user_manual.rst b/doc/user_manual.rst new file mode 100644 index 000000000..049f3759d --- /dev/null +++ b/doc/user_manual.rst @@ -0,0 +1,883 @@ +PTXdist User’s Manual +===================== + +This chapter should give any newbie the information he/she needs to be +able to handle any embedded Linux projects based on PTXdist. Also the +advanced user may find new valueable information. + +How does it work? +----------------- + +PTXdist supports various aspects of the daily work to develop, deploy +and maintain an embedded Linux based project. + +.. figure:: figures/project-handling.png + :alt: Objectives in a project + [fig:all:sub:`p`\ rojects\ :sub:`a`\ spects] + + Objectives in a project [fig:all:sub:`p`\ rojects\ :sub:`a`\ spects] + +The most important part is the development. For this project phase, +PTXdist provides features to ensure reproducibility and verifiability. + +PTXdist’s perception of the world +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PTXdist works project centric. A PTXdist project contains all +information and files to populate any kind of target system with all +required software components. + +- Specific configuration for + + - Bootloader + + - Kernel + + - Userland (root filesystem) + +- Adapted files (or generic ones) for runtime configuration + +- Patches for all kind of components (to fix bugs or improve features) + +Some of these information or files are coming from the PTXdist base +installation (patches for example), but also can be part of the project +itself. By this way, PTXdist can be adapted to any kind of requirement. + +Most users are fine with the information and files the PTXdist base +installation provides. Development of PTXdist is done in a way to find +default settings most user can work with. But advanced users can still +adapt to their special needs. + +As stated above, a PTXdist project consists of all required parts, some +of these parts are separated by design: PTXdist separates a platform +configuration from userland configuration (root filesystem). So, +platforms can share a common userland configuration, but use a specific +kernel configuration in their own platform configuration. + +Collecting various platforms into one single project should help to +maintain such projects. But some platforms do need special userland +(think about graphic/non graphic platforms). To be able to also collect +this requirement into one single project, so called *collections* are +supported. With this feature, a user can configure a full featured main +userland, reduced via a collection by some components for a specific +platform where it makes no sense to build and ship them. + +A different use case for collections could be the security of an +application. While the development is ongoing all kind of debugging and +logging helpers are part of the root filesystem. But the final +production root filesystem uses collections to omit all these helpers +and to reduce the risc of security vulnerability. + +PTXdist can handle the following project variations: + +- one hardware platform, one userland configuration (common case) + +- one hardware platform, various userland configurations + +- various hardware platforms, one userland configuration (common case) + +- various hardware platforms, one userland configuration, various + collections + +- various hardware platforms, various userland configuration + +- various hardware platforms, various userland configuration, various + collections + +PTXdist’s build process +~~~~~~~~~~~~~~~~~~~~~~~ + +When PTXdist is building one part (we call it a *package*)of the whole +project, it is divided into up to six stages: + +.. figure:: figures/ptxbuild + :alt: The build process [fig:the:sub:`b`\ uild\ :sub:`p`\ rocess] + + The build process [fig:the:sub:`b`\ uild\ :sub:`p`\ rocess] + +get + The package will be obtained from its source (downloaded from the + web for example) + +extract + The package archive gets extracted and patched if a patch set for + this package exists + +prepare + Many packages can be configured in various ways. If supported, this + stage does the configuration in a way defined in the menu (project + specific) + +compile + The package gets built. + +install + The package installs itself into a project local directory. This + step is important at least for libraries (other packages may depend + on) + +targetinstall + Relevant parts of the package will be used to build an IPKG archive + and the root filesystem + +For each single package, one so called *rule file* exists, describing +the steps to be done in each stage shown above (refer section +[sect:rulefile:sub:`l`\ ayout] for further details). + +Due to the *get* stage, PTXdist needs a working internet connection to +download an archive currently not existing on the development host. But +there are ways to prevent PTXdist from doing so (refer to section +[subsection:source:sub:`a`\ rchives]). + +First steps with PTXdist +------------------------ + +PTXdist works as a console command tool. Everything we want PTXdist to +do, we have to enter as a command. But it’s always the same base +command: + +:: + + $ ptxdist |\textless{}parameter\textgreater{}| + +To run different functions, this command must be extended by parameters +to define the function we want to run. + +If we are unsure what parameter must be given to obtain a special +function, we run it with the parameter *help*. + +:: + + $ ptxdist help + +This will output all possible parameters ans subcommands and their +meaning. + +As the list we see is very long, let’s explain the major parameters +usually needed for daily usage: + +menu + This starts a dialog based frontend for those who do not like typing + commands. It will gain us access to the most common parameters to + configure and build a PTXdist project. + +menuconfig + Starts the Kconfig based project configurator for the current + selected userland configuration. This menu will give us access to + various userland components that the root filesystem of our target + should consist of. + +menuconfig platform + Starts the Kconfig based platform configurator. This menu lets us + set up all target specific settings. Major parts are: + + - Toolchain (architecture and revision) + + - boot loader + + - root filesystem image type + + - Linux kernel (revision) + + Note: A PTXdist project can consist of more than one platform + configuration at the same time. + +menuconfig kernel + Runs the standard Linux kernel Kconfig to configure the kernel for + the current selected platform. To run this feature, the kernel must + be already set up for this platform. + +menuconfig collection + If multiple platforms are sharing one userland configuration, + collections can define a subset of all selected packages for + specific platforms. This is an advanced feature, rarely used. + +toolchain + | Sets up the path to the toolchain used to compile the current + selected platform. Without an additional parameter, PTXdist tries + to guess the toolchain from platform settings. To be successful, + PTXdist depends on the OSELAS.Toolchains installed to the ``/opt`` + directory. + | If PTXdist wasn’t able to autodetect the toolchain, an additional + parameter can be given to provide the path to the compiler, + assembler, linker and so on. + +select + Used to select the current userland configuration, which is only + required if there is no ``selected_ptxconfig`` in the project’s main + directory. This parameter needs the path to a valid ``ptxconfig``. + It will generate a soft link called ``selected_ptxconfig`` in the + project’s main directory. + +platform + Used to select the current platform configuration, which is only + required if there is no ``selected_platformconfig`` in the project’s + main directory. This parameter needs the path to a valid + ``platformconfig``. It will generate a soft link called + ``selected_platformconfig`` in the project’s main directory. + +collection + Used to select the current collection configuration, which is only + required in special cases. This parameter needs the path to a valid + ``collection``. It will generate a soft link called + ``selected_collection`` in the project’s main directory. This is an + advanced feature, rarely used. + +go + The mostly used command. This will start to build everything to get + all the project defined software parts. Also used to rebuild a part + after its configuration was changed. + +images + Used at the end of a build to create an image from all userland + packages to deploy the target (its flash for example or its hard + disk). + +setup + Mostly run once per PTXdist revision to set up global paths and the + PTXdist behavior. + +All these commands depending on various files a PTXdist based project +provides. So, running the commands make only sense in directorys that +contains a PTXdist based project. Otherwise PTXdist gets confused and +confuses the user with funny error messages. + +To show the usage of some listed major subcommands, we are using a +generic PTXdist based project. + +Extracting the Board Support Package +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In order to work with a PTXdist based project we have to extract the +archive first. + +:: + + $ tar -zxf |\verbatimcmd\ptxdistBSPName|.tar.gz + $ cd |\verbatimcmd\ptxdistBSPName| + +PTXdist is project centric, so now after changing into the new directory +we have access to all valid components. + +Notes about some of the files and directories listed above: + +ChangeLog + Here you can read what has changed in this release. Note: This file + does not always exist. + +documentation + If this BSP is one of our OSELAS BSPs, this directory contains the + Quickstart you are currenly reading in. + +configs + A multiplatform BSP contains configurations for more than one + target. This directory contains the respective platform + configuration files. + +projectroot + Contains files and configuration for the target’s runtime. A running + GNU/Linux system uses many text files for runtime configuration. + Most of the time, the generic files from the PTXdist installation + will fit the needs. But if not, customized files are located in this + directory. + +rules + If something special is required to build the BSP for the target it + is intended for, then this directory contains these additional + rules. + +patches + If some special patches are required to build the BSP for this + target, then this directory contains these patches on a per package + basis. + +tests + Contains test scripts for automated target setup. + +Next we will build the to show some of PTXdist’s main features. + +This part describes how to handle a ready-to-use BSP. + +Userland can be more generic than a kernel and – for example – a boot +loader. Due to this it’s possible to use one userland configuration for +various hardware platforms provided by one BSP. PTXdist defines a +userland configuration by selecting a config file that contains all the +required settings. + +The selecting step offers the possibility to provide more than one +predefined configuration. We can use one of them on demand, whenever we +are going to build the BSP. This is intended for special cases where the +collection feature does not meet all requirements. + +So a PTXdist BSP can provide various combinations: + +- one hardware platform, one software platform + +- one hardware platform, various software platforms + +- various hardware platforms, one software platform + +- various hardware platforms, one software platform, various + collections + +- various hardware platforms, various software platforms + +- various hardware platforms, various software platforms, various + collections + +Every combination supports a special case of requirements. It’s up to +the user what combination meets the project needs. + +Keeping all hardware platforms in one BSP could decrease the maintenance +overhead. Using one software platform for all hardware platforms also. +For special cases where hardware platforms differ in main features, +collections could help to continue using one software platform, by +switching on or off special applications on a per use-case base. + +Selecting a Userland Configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +First of all we have to select a userland configuration. This step +defines what kind of applications will be built for the hardware +platform. The comes with a predefined configuration we select in the +following step: + +:: + + $ ptxdist select configs/ptxconfig + info: selected ptxconfig: + |\hspace*{1cm}|'configs/ptxconfig' + +Selecting a Hardware Platform +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Before we can build this BSP, we need to select one of the possible +platforms to build for. In this case we want to build for the : + +:: + + $ ptxdist platform configs/|\verbatimcmd\ptxdistPlatformName|/platformconfig|\verbatimcmd\ptxdistPlatformVariant| + info: selected platformconfig: + |\hspace*{1cm}| 'configs/|\ptxdistPlatformName|/platformconfig|\ptxdistPlatformVariant|' + +Note: If you have installed the OSELAS.Toolchain() at its default +location, PTXdist should already have detected the proper toolchain +while selecting the platform. In this case it will output: + +:: + + found and using toolchain: + '/opt/OSELAS.Toolchain-|\oselasTCNVendorVersion \oselasTCNVendorPatchLevel|/|\ptxdistCompilerName|/|\ptxcr| + |\hspace{20pt}\ptxdistCompilerVersion|/bin' + +If it fails you can continue to select the toolchain manually as +mentioned in the next section. If this autodetection was successful, we +can omit the steps of the section and continue to build the BSP. + +In the unified , one included platform can use more userland features +than another. For example platforms with graphic features will also +build graphic support, but platforms sans display do not need it. To +speed up compilation for specific platforms PTXdist provides +collections, to reduce the amount of programs to be compiled for +specific cases. + +To reduce the package count for the run: + +:: + + $ ptxdist collection configs/|\verbatimcmd\ptxdistPlatformCollection| + info: selected collectionconfig: + |\hspace*{1cm}|'configs/|\ptxdistPlatformCollection|' + +Selecting a Toolchain +~~~~~~~~~~~~~~~~~~~~~ + +If not automatically detected, the last step in selecting various +configurations is to select the toolchain to be used to build everything +for the target. + +:: + + $ ^ptxdist toolchain /opt/OSELAS.Toolchain-|\verbatimcmd\oselasTCNVendorVersion \oselasTCNVendorPatchLevel|/|\verbatimcmd\ptxdistCompilerName|/|\verbatimcmd\ptxcr| + |\hspace*{1cm}\verbatimcmd\ptxdistCompilerVersion|/bin + +Building the Root Filesystem Content +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Now everything is prepared for PTXdist to compile the BSP. Starting the +engines is simply done with: + +:: + + $ ptxdist go + +PTXdist does now automatically find out from the ``selected_ptxconfig`` +and ``selected_platformconfig`` files which packages belong to the +project and starts compiling their *targetinstall* stages (that one that +actually puts the compiled binaries into the root filesystem). While +doing this, PTXdist finds out about all the dependencies between the +packages and builds them in correct order. + +What we Got Now +~~~~~~~~~~~~~~~ + +After building the project, we find even more sub directories in our +project. + +build-cross + Contains all packages sources compiled to run on the host and handle + target architecture dependend things. + +build-host + Contains all packages sources compiled to run on the host and handle + architecture independend things. + +build-target + Contains all package sources compiled for the target architecure. + +images + Generated files for the target can be found here: Kernel image and + root filesystem image. + +packages + Location for alle individual packages in ipk format. + +sysroot-target + Contains everything target architecture dependend (libraries, header + files and so on). + +sysroot-cross + Contains everything that is host specific but must handle target + architecture data. + +sysroot-host + Contains everything that is only host specific. + +root + | Target’s root filesystem image. This directory can be mounted as + an NFS root for example. + +root-debug + | Target’s root filesystem image. The difference to ``root/`` is, + all programs and libraries in this directory still have their + debug information present. This directory is intended to be used + as system root for a debugger. To be used by the debugger, you + should setup your debugger with + | ``set solib-absolute-prefix /root-debug`` + +state + Building every package is divided onto stages. And stages of one + package can depend on stages of other packages. In order to handle + this correctly, this directory contains timestamp files about + finished stages. + +This are the generated files: + +logfile + Every run of PTXdist will add its output to this file. If something + fails, this file can help to find the cause. + +Creating a Root Filesystem Image +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +After we have built the root filesystem content, we can make an image, +which can be flashed to the target system or copied on some kind of disk +media. To do so, we just run + +:: + + $ ptxdist images + +PTXdist now extracts the content of priorly created *\*.ipk* packages to +a temporary directory and generates an image out of it. PTXdist supports +following image types: + +**hd.img:** contains bootloader, kernel and root files in an ext2 +partition. Mostly used for X86 target systems. + +**root.jffs2:** root files inside a jffs2 filesystem. + +**uRamdisk:** a u-boot loadable Ramdisk + +**initrd.gz:** a traditional initrd RAM disk to be used as initrdramfs +by the kernel + +**root.ext2:** root files inside an ext2 filesystem. + +**root.squashfs:** root files inside a squashfs filesystem. + +**root.tgz:** root files inside a plain gzip compressed tar ball. + +All these files can be found in ``images`` if enabled. + +Running all Parts in an emulated Environment (QEMU) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The is prepared to give every user a chance to run the results of the +previous steps even in the absense of real hardware. All we need is a +working QEMU on our development host. + +Simply run + +:: + + |\$ ./configs/\ptxdistPlatformName{}/run| + +This will start QEMU in full system emulation mode and runs the +previously built kernel which then uses the generated disk image to +bring up a full Linux based system. + +The running system uses a serial device for its communication. QEMU +forwards this emulated device to the current development host console. +So, we can watch the starting kernel’s output and log in on this system. + +Note: Log in as user ’\ ``root``\ ’ with no password (just enter). + +Also a telnet deamon is running in this emulation. QEMU is configured to +forward the standard telnet port 23 of the emulated system to host’s +port 4444. To connect to the emulated system, we can just run a + +:: + + $ telnet localhost 4444 + Trying 127.0.0.1... + Connected to localhost. + Escape character is '^]'. + + ptx login: root + root@ptx:~ + +Leaving the emulated environment happens by entering the key sequence +*CTRL-A X* in the development host console. + +Adapting the Project +-------------------- + +Handling a fully prepared PTXdist project is easy. But everything is +fixed to the settings the developer selected. We now want to adapt the +project in a few simple settings. + +Working with Kconfig +~~~~~~~~~~~~~~~~~~~~ + +Whenever we modify our project, PTXdist is using *Kconfig* to manipulate +the settings. *Kconfig* means *kernel configurator* and was mainly +developed to configure the Linux kernel itself. But it is easy to adapt, +to use and so popular that more and more projects are using *Kconfig* +for their purposes. PTXdist is one of them. + +What is Kconfig +^^^^^^^^^^^^^^^ + +It is a user interface to select given resources in a convenient way. +The resources that we can select are given in simple text files. It uses +a powerful “language” in these text files to organize them in a +hierarchical manner, solves challenges like resource dependencies, +supports help and search features. PTXdist uses all of these features. +*Kconfig* supports a text based user interface by using the *ncurses* +library to manipulate the screen content and should work on nearly all +host systems. + +For example running PTXdist’s ``menuconfig`` subcommand in this way + +:: + + $ ptxdist menuconfig + +will show the following console output + +.. figure:: figures/menuconfig_intro + :alt: Main userland configuration menu + [fig:main:sub:`u`\ serland\ :sub:`m`\ enu] + + Main userland configuration menu + [fig:main:sub:`u`\ serland\ :sub:`m`\ enu] + +Navigate in Kconfig menu (select, search, ...) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To navigate through the configuration tree, we are using the arrow keys. +Up and down navigates vertically in the menu entries. Right and left +navigates between *Select*, *Exit* and *Help* (in the bottom part of our +visual screen). + +To enter one of the menus, we navigate to this entry to highlight it and +press the *Enter* key. To leave it, we select *Exit* and press the +*Enter* key again. There are shortcuts available, instead of pressing +the *Enter* key to enter a menu we also can press *alt-s* and to leave a +menu *alt-e*. Also an ESC double hit leaves any menu we are in. + +To select a menu entry, we use the *Space* key. This will toggle the +selection. Or, to be more precise and faster, we use the key *y* to +select an entry, and key *n* to deselect it. + +To get help for a specific menu topic, we navigate vertically to +highlight it and horizontally to select the *Help* entry. Then we can +press *Enter* to see the help. + +To search for specific keywords, we press the */* key and enter a word. +Kconfig then lists all occurences of this word in all menus. + +Meaning of visual feedbacks in Kconfig +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- | Submenus to enter are marked with a trailing ``--->`` + | Note: Some submenus are also marked with a leading bracket ``[ ]``. + To enter them we first must select/enable them ``[*]`` + +- Entries with a list of selectable alternatives are also marked with a + trailing ``--->`` + +- Entries we can select are marked with a leading empty bracket ``[ ]`` + +- Entries that are already selected are marked with a leading filled + bracket ``[*]`` + +- Entries that are selected due to dependencies into other selected + entries are marked with a leading ``-*-`` + +- Some entries need a free text to enter, they are marked with leading + brackets ``()`` and the free text in it + +Menus and submenus in Kconfig (sectioning) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are dozens of entries in the PTXdist configuring menus. To handle +them, they are divided and separated into logical units. + +The main building blocks in the *userland configuration* menu are: + +- Host Options: Some parts of the project are build host relevant only. + For example PTXdist can build the DDD debugger to debug applications + running on the target. + +- Root Filesystem: Settings to arrange target’s root filesystem and to + select the main C runtime library + +- Applications: Everything we like to run on your target. + +The main building blocks in the *platform configuration* menu are: + +- Architecture: Basic settings, like the main and sub architecture the + target system uses, the toolchain to be used to build everything and + some other architecture dependent settings. + +- Linux kernel: Which kernel revision and kernel configuration should + be used + +- Bootloader: Which bootloader (if any) should be built in the project + +- The kind of image to populate a root filesystem into the target + system + +The main building blocks in the *board setup configuration* menu are: + +- Network: Network settings for the target + +- Host: Host setup to be able to reach the target system + +At this point it could be useful to walk to the whole menus and their +submenus to get an idea about the amount of features and applications +PTXdist currently supports. + +Adapting Platform Settings +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some parts of the project are platform specific (in contrast to the +userland configuration that could be shared between platforms). We now +want to change the used Linux kernel of our current platform. It comes +with a default linux-3.0 and we want to change it to a more recent +linux-3.7. + +To do so, we run: + +:: + + $ ptxdist menuconfig platform + +In this Kconfig dialogue we navigate to the entry: + +:: + + Linux kernel ---> + (|\ptxdistPlatformKernelRev{}|) kernel version + +and replace the 3.0 value by the 3.7 value. + +Since PTXdist checks the MD5 sums of the archives it uses, we also must +change the MD5 sum in the menu entry according to the selected kernel +version. + +Use one of the following MD5 sums for a kernel of your choice: + +- 3.7: ``21223369d682bcf44bcdfe1521095983`` + +- 3.6: ``1a1760420eac802c541a20ab51a093d1`` + +- 3.5: ``24153eaaa81dedc9481ada8cd9c3b83d`` + +- 3.4: ``967f72983655e2479f951195953e8480`` + +- 3.3: ``7133f5a2086a7d7ef97abac610c094f5`` + +- 3.2: ``364066fa18767ec0ae5f4e4abcf9dc51`` + +- 3.1: ``8d43453f8159b2332ad410b19d86a931`` + +- 3.0: ``398e95866794def22b12dfbc15ce89c0`` + +- 2.6.39: ``1aab7a741abe08d42e8eccf20de61e05`` + +- 2.6.38: ``7d471477bfa67546f902da62227fa976`` + +- 2.6.37: ``c8ee37b4fdccdb651e0603d35350b434`` + +Now we can leave the menu and save the new settings. + +A Linux kernel needs a configuration for being built correctly. The +project comes with a prepared configuration in the file +``configs//kernelconfig-3.0`` for the 3.0 kernel. + +It is always a good idea to start with a known-to-work kernel +configuration. So, for this example, we are using a different +known-to-work kernel configuration in the ``configs//kernelconfig-3.7`` +file for our new 3.7 kernel. + +Adapting Linux Kernel Settings +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In this section we want to show how to change some Linux kernel settings +of our project. + +First of all, we run + +:: + + $ ptxdist menuconfig kernel + +This command will start the kernel’s Kconfig. For this example we want +to enable USB host support in the kernel. To do so, we navigate to: + +:: + + Device Drivers ---> + [ ] USB support ---> + < > Support for Host-side USB + < > OHCI HCD support + +Note: All the listed empty ``[ ]`` and ``< >`` above must be activated +to get all submenu entries. + +We leave the menu and save the new kernel configuration. + +To start building a new kernel with the new configuration, we again run: + +:: + + $ ptxdist go + +This builds or re-builds the kernel, because we changed its settings. + +Note: If nothing was changed, ``ptxdist go`` also will do nothing. + +When PTXdist has finished its job, the new bootable kernel can be found +at ``images/linuximage``. To boot it again in the QEMU emulation, the +hard disk image must be re-created with: + +:: + + $ ptxdist images + |\$ ./configs/\ptxdistPlatformName/run| + +The emulated system should now start with a 3.7 based kernel with USB +support. + +Adapting Userland Settings +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +After changing some platform and kernel settings, we are now reaching +the most interesting area: Userland. + +In the userland area we can enable and use all the applications and +services PTXdist provides. Many of them are working out of the box when +enabled and executed on the target side. Some need additional runtime +configuration, but PTXdist comes with most common configurations for +such packages. + +In this simple example, we want to add the missing ``head`` command to +our target’s shell. Assuming we forgot to enable this command, we get: + +:: + + |\$ ./configs/\ptxdistPlatformName/run| + + ptx login: root + login[xxx]: root login on 'ttyS0' + root@ptx:~ head /etc/fstab + -sh: head: not found + +To change this, we first run: + +:: + + $ ptxdist menuconfig + +The additional command we want to enable is provided by the *Busybox* +package. So we navigate to: + +:: + + Shell & Console Tools ---> + -*- busybox ---> + Coreutils ---> + [ ] head + +After activating the ``[ ] head`` entry, we leave the menu and save the +new configuration. + +Once again, a + +:: + + $ ptxdist go + +will build or re-build the busybox package due to its configuration +change. + +And also once again, after finishing its job, the following commands let +us test the new command: + +:: + + $ ptxdist images + |\$ ./configs/\ptxdistPlatformName/run| + +Log in on the emulated system and simply check with a: + +:: + + ptx login: root + login[xxx]: root login on 'ttyS0' + root@ptx:~ head /etc/fstab + # + # /etc/fstab + # + + # special filesystems + proc /proc proc defaults 0 0 + debugfs /sys/kernel/debug debugfs defaults,noauto 0 0 + devpts /dev/pts devpts defaults 0 0 + none /tmp tmpfs defaults,mode=1777,uid=0,gid=0 0 0 + none /sys sysfs defaults 0 0 + +We are done now. These simple examples should give the users a quick +feeling how things are working in PTXdist and how to modify them. +Adapting this generic BSP to a different platform with nearly the same +features as our reference platforms is possible with this knowledge. + +But most of the time, a user needs more detailed adaptions to be able to +fit all requirements of the new platform. At this point of time we are +no longer ordinary users of PTXdist, we become developers now. + +So, right now it’s time to read the *PTXdist Developer’s Manual* + diff --git a/doc/welcome.rst b/doc/welcome.rst new file mode 100644 index 000000000..9d33c1bd0 --- /dev/null +++ b/doc/welcome.rst @@ -0,0 +1,133 @@ +Welcome to the Embedded World +============================= + +First Steps in the Embedded World +--------------------------------- + +Once upon in time, programming embedded systems was easy: all a +developer needed when he wanted to start a new product was a good +toolchain, consisting of + +a compiler + +maybe an assembler + +probably an EPROM burning device + +and things could start. After some more or less short time, every +register of the CPU was known, a variety of library routines had been +developed and our brave developer was able to do his project with the +more and more well-known system. The controllers had legacy interfaces +like RS232, i2c or SPI which connected them to the outside world and the +main difference between the controllers available on the market was the +number of GPIO pins, UARTS and memory ressources. + +Things have changed. Hardware manufacturers have weakened the border +between deeply embedded microcontrollers – headless devices with just a +few pins and very limited computing power – and full blown +microprocessors. System structures became much more complicated: where +our good old controllers have had just some interrupts with some small +interrupt service routines, we today need complicated generic interrupt +infrastructures, suitable for generic software frameworks. Where we’ve +had some linearly mapped flash ROM and some data RAM we today have +multi-stage-pipeline architectures, memory management units, virtual +address spaces, on-chip-memory, caches and other complicated units, +which is not exactly what the embedded system developer wants program +every other day. + +Entering embedded operating systems. Although there are still some +processors out there (like the popular ARM7TDMI based SoCs) which can be +programmed the good old non-operating-system way with reasonable effort, +it in fact is becoming more and more difficult. On the other hand, +legacy I/O interfaces like RS232 are increasingly often replaced by +modern plug-and-play aware communication channels: USB, FireWire +(IEEE1394), Ethernet & friends are more and more directly being +integrated into today’s microcontroller hardware. Whereas some of these +interfaces can “somehow” be handled the old controller-style way of +writing software, the developer following this way will not be able to +address the security and performance issues which come up with the +modern network accessible devices. + +During the last years, more and more of the small-scale companies which +developed little embedded operating systems have been pushed out of the +market. Nearly no small company is able to support all the different +interfaces, communication stacks, development tools and security issues +out there. New interfaces and -variants (like USB On-the-Go) are +developed faster than operating system developers can supply the +software for them. The result is a consolidation of the market: today we +see that, besides niche products, probably only the largest commercial +embedded operating system suppliers will survive that development. + +Only the largest commercial...? There is one exception: when the same +situation came up in the “mainstream” computer market at the beginning +of the 1990ies, people started to develop an alternative to the large +commercial operating systems: Linux. Linux did never start with a +ready-to-use solution: people had a problem, searched for a solution but +didn’t find one. Then they started to develop one themselves, often +several people did this in parallel, and in a huge community based +evolution mechanism the best solutions found their way into the Linux +kernel, which over the time formed one of the most reliable and +performant kernels available today. This “develop-and-evolute” mechanism +has shown its effectiveness over and over again in the server and +desktop market of today. + +From Server to Embedded +----------------------- + +The fact that for most technical problems that might occur it may be +possible to find somebody on the internet who has already worked on the +same or another very similar problem, was one of the major forces behind +the success story of Embedded Linux. + +Studies have shown that more than 70% of the embedded developers are not +satisfied with a black-box operating system: they want to adapt it to +their needs, to their special hardware situation (which most times is +Just Different than anything available). Embedded projects are even more +variegated than desktop- or server projects, due to the fact that there +exist so many different embedded processors with lots of peripherals out +there. + +Linux has evolved from an i386 only operating system to a kernel running +on nearly every modern 32 bit processor available today: x86, PowerPC, +ARM, MIPS, m68k, cris, Super-H etc. The kernel supplies a hardware +abstraction layer which lets our brave embedded developer once again +concentrate on his very special problem, not on handling negligibilities +like memory management. + +But Linux is only half of the story. Besides the kernel, a Linux based +embedded system consists of a “userland”: a filesystem, containing all +the small tools which form a small Unix system. Only the combination of +the kernel and a Userland let’s the developer run “normal” processes on +his x86 development machine as well as on his embedded target. + +Linux = Embedded Linux +---------------------- + +Whereas the mainstream developers were always able to use normal Linux +distributions like SuSE, RedHat, Mandrake or Debian as a base for their +applications, things are different for embedded systems. + +Due to the restricted ressources these systems normally have, +distributions have to be small and should only contain those things that +are needed for the application. Today’s mainstream distributions cannot +be installed in less than 100 MiB without major loss of functionality. +Even Debian, probably today the most customizable mainstream +distribution, cannot be shrunk below this mark without for example +losing the packet management, which is an essential feature of using a +distribution at all. + +Additionally, source code for industrial systems has to be + +auditable and + +reproducible. + +Embedded developers usually want to know what’s in their systems – be it +that they have to support their software for a long time span (something +like 10-15 years are usual product lifetimes in automation applications) +or that they have such a special scenario that they have to maintain +their integrated source of their Userland themselves. + +**Entering PTXdist.** + + -- cgit v1.2.3