Opened 3 years ago

Closed 4 weeks ago

Last modified 4 weeks ago

#1253 closed enhancement/feature request (fixed)

Add build scripts from build server to Navit SVN

Reported by: mvglasow (2) Owned by: KaZeR
Priority: trivial Milestone: version 0.5.1
Component: tools Version: git master
Severity: normal Keywords: build
Cc: http://wiki.navit-project.org/index.php/user:mvglasow, (2), bart.cockheyt@…

Description

The Navit build server, which prepares the binaries available at download.navit-project.org, appears to use scripts that do a completely automated build.

In particular, it supplies all the parameters needed to build a particular flavor of Navit, which is of particular interest for the more exotic ones such as android-x86 or android-armv4.

I suggest making these available in the SVN code tree. This would simplify building the more exotic flavors, as well as ensuring consistency between private builds and downloaded binaries. (I've had quite a few instances in which I was unable to build versions/flavors of navit that would build on the server, and being able to retry the build with the exact same parameters as the server would have been a lot of help.)

Change History (14)

comment:1 Changed 3 years ago by mvglasow (2)

  • Cc http://wiki.navit-project.org/index.php/user:mvglasow (2) added

comment:2 Changed 3 years ago by sleske

Excellent idea. This would also make the maintenance of these scripts easier, which would help in cases like #1144, where the build needs to be changed.

comment:3 Changed 3 years ago by kazer

I actually see a couple of issues with the current build scripts :

  • they have mostly been developed by cp15, and they don't include a license header. I don't think cp15 would mind much, but I don't want to publish his work without his express consent
  • the build scripts heavily depend of the environment that has been setup on the build server. AFAIK, it has been a manual process
  • some of the features might require a license for a 3rd party tool ( I'm thinking about the windows build, but I'm not sure )
  • iirc, there isn't a lot of tasks parallelization

On the other hand :

  • we can probably restart this effort from scratch in a collaborative manner
  • I'd love to use something like jenkins or travis for that. Apart from building, we could start doing some QA and automated testing
  • i've started some work using docker to build specifically tailored build environments that would be easier to maintain

I am still wondering if this should be shipped as a part of Navit or if we should host it somewhere else ( because theses scripts would be of no interest for end-users or package maintainers).

If anybody would like to help, feel free to chime in.

comment:4 Changed 3 years ago by mvglasow (2)

All right, then:

  • let's ask cp15 for consent to GPL his build scripts
  • put them up for review (initially, this can be somewhere outside the navit code tree, until we reach an agreement on how to proceed)
  • see how much is specific to the build environment, and if there are any dependencies on proprietary code
  • even if the scripts aren't quite mature yet, that shouldn't prevent us from putting them in the code tree – that is quite likely to help them mature

As for dependency on the build server setup – even in that case the scripts can serve as example code, and once they are available to all contributors, more people have the chance to turn them into something generic. At the very worst, declare them sample code.

As for dependencies on proprietary tools – I remember building for Windows CE without needing any proprietary software. Even if there are any such dependencies – that shouldn't be an obstacle to publishing the script (not including, obviously, the proprietary software itself).

Automated building with jenkins sounds great, however for me the main motivation is to be able to build with the same parameters as the Navit build server. If it builds on the server but not on my machine, I know there's something wrong with my setup; if the server build failed as well, I know the error is not on my part. Back in the WinCE days, I had builds fail while they worked on the server, and could not figure out whether that was due to an incorrect build parameter, different versions of the toolchain or something else.

Eventually I'd tend towards shipping the build scripts with the code tree. They are probably of little interest to end users, but then again, end users would get a binary distribution rather than the code. They also may not be of much use for package maintainers – but there's already more in the source tree that package maintainers wouldn't need (basically anything that is specific to platforms other than pure Linux). But it's potentially interesting for anyone who wants to build navit on their own machine – not being able to build Navit may put off some would-be contributors.

In conclusion: with cp15's consent, having the build scripts available is more beneficial than not having them.

comment:5 Changed 3 years ago by tryagain

Actually build logs at http://download.navit-project.org/logs/navit/stats.html already contain everything needed to reproduce the build process, even program paths can be deducted.

We could put the scripts into the svn trees, but there will be probably not much more than a single line containing some suggested cmake (or "configure") options.

Also packaging for some systems is not that trivial. But that probably should anyway be moved into svn as part of cmake config files.

comment:6 Changed 3 years ago by bafplus

  • Cc bart.cockheyt@… added

comment:8 follow-up: Changed 2 years ago by kazer

We made a lot of progress on the automation and we now build automatically for Linux/Windows/Android? using CircleCI. The scripts are versioned with the code too and we even have automatic updates to Google Play Store.

Apart from the iPhone build that will require specific work, are we currently missing a platform that we want to maintain?

comment:9 Changed 2 years ago by mvglasow (2)

Is the WinCE port still maintained? (Is it compatible with recent versions of Windows Phone at all?) I remember building was tricky, so having CircleCI as a working toolchain would help if we're still maintaining that port. (I ditched Windows Mobile in favor of Android years ago, and also Windows in favor of Linux, thus I haven't followed the last few years of Windows development.)

comment:10 in reply to: ↑ 8 Changed 2 years ago by sleske

Replying to http://wiki.navit-project.org/index.php/user:kazer:

Apart from the iPhone build that will require specific work, are we currently missing a platform that we want to maintain?

Well, there are quite a few other platforms that Navit built on at some point.

If I understand this correclty, CirclCI currently builds for

  • Linux/x64
  • Android/arm
  • Windows/x86

The old build server ( http://download.navit-project.org/logs/navit/stats.html ) lists successful builds for

  • iPhone
  • Linux
  • Win32
  • WinCE

plus failed builds for:

The Navit wiki ( http://wiki.navit-project.org/index.php/Main_Page ) also lists various Mobile Linux platforms. So I think, apart from the iPhone, the Mobile Linux platforms are the only ones missing from CircleCI.

Having these build scripts _might_ be helpful - on the other hand, I don't know whether there is still interest in these platforms. Also, the failing builds seem to still use the old autotools build, which will (should) probably go away soon.

So, if we can store the build scripts somewhere, I'd be all for it, but if it's not practical, I think we can close this ticket.

comment:11 Changed 2 years ago by mvglasow (2)

...I knew I'd forgotten something. Android/x86, of course – we should have that one. It would make Android testing a lot easier (Android x86 in a VirtualBox? is way faster than the official emulator).

comment:12 Changed 3 months ago by http://wiki.navit-project.org/index.php/user:jkoan

  • Milestone set to version 0.5.1

comment:13 Changed 4 weeks ago by http://wiki.navit-project.org/index.php/user:jkoan

  • Resolution set to fixed
  • Status changed from new to closed

All current buildscripts are on the repo which used on circleci. If builds are missing we need to write new scripts.

comment:14 Changed 4 weeks ago by kazer

Well the wince build is still missing but I might have been able to rope number6 in having a look at it. Fingers crossed :)

Note: See TracTickets for help on using tickets.