patchFreeciv - Patches: patch #1391, [Metaticket] new code to generate...

 
 
Show feedback again

patch #1391: [Metaticket] new code to generate stats (graphics)

Submitted by:  Matthias Pfafferodt <syntron>
Submitted on:  Sat 19 Dec 2009 11:23:08 PM UTC  
 
Category: generalPriority: 3 - Low
Status: DonePrivacy: Public
Assigned to: Matthias Pfafferodt <syntron>Open/Closed: Closed
Planned Release: 2.4.0

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sat 26 Feb 2011 03:39:33 PM UTC, comment #24:

> (does anybody uses this?)


I had a play with it before it was committed to trunk; it's pretty cool. Now I'm trying it again on trunk to monitor an autogame (previously I used --Ppm for this).

One thing it could do with is some way to specify which directory server-created image files are written to (currently they go in the working directory), but that should be fairly easy to add.

Jacob Nevins <jtn>
Project Administrator
Thu 28 Oct 2010 01:18:55 PM UTC, comment #23:

This patch series is now targeted for freeciv 2.4.

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Wed 20 Oct 2010 11:48:01 AM UTC, comment #22:

updated all-in-one patch

(does anybody uses this?)

(file #10847)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Wed 13 Oct 2010 06:03:13 PM UTC, comment #21:

updated all-in-one patch

(file #10766)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Tue 12 Oct 2010 01:14:52 PM UTC, comment #20:

The last message shouldn't be there; here is the complete comment:

I did update the mapimg patches fixing all the TODO's. At the moment help is needed for the sdl and gtk client (see patch #2061). Attached to this ticket is a file including all the patches listed below.

[01/12] patch #2057: [mapimg] rename registry.c:MAX_LEN_BUFFER to MAX_LEN_ERRORBUF
[02/12] patch #2058: [mapimg] add rgbcolor.(ch)
[03/12] patch #2059: [mapimg] update colors_common to use struct rgbcolor
[04/12] patch #2060: [mapimg] define terrain colors in the ruleset
[05/12] patch #2061: [mapimg] define player colors in the ruleset
[06/12] patch #2062: [mapimg] new setting playercolor + command /playercolor
[07/12] patch #2063: [mapimg] update display for 'list players'
[08/12] patch #2018: [mapimg] implement map image
[09/12] patch #2019: [mapimg] map image support for the client
[10/12] patch #2020: [mapimg] script to generate animated gifs from map images
[11/12] patch #2021: [mapimg] add magickwand as another toolkit for the creation of map images
[12/12] patch #1928: [mapimg] remove save_ppm and -P option

(file #10749)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Tue 12 Oct 2010 01:09:50 PM UTC, comment #19:

[01/12] patch #2057: [mapimg] rename registry.c:MAX_LEN_BUFFER to MAX_LEN_ERRORBUF
patch #2058: [mapimg] add rgbcolor.(ch)
patch #2059: [mapimg] update colors_common to use struct rgbcolor
patch #2060: [mapimg] define terrain colors in the ruleset
patch #2061: [mapimg] define player colors in the ruleset
patch #2062: [mapimg] new setting playercolor + command /playercolor
patch #2063: [mapimg] update display for 'list players'
patch #2018: [mapimg] implement map image
patch #2019: [mapimg] map image support for the client
patch #2020: [mapimg] script to generate animated gifs from map images
patch #2021: [mapimg] add magickwand as another toolkit for the creation of map images
patch #1928: [mapimg] remove save_ppm and -P option

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sun 03 Oct 2010 04:50:11 PM UTC, comment #18:

This is now a meta ticket for the mapimg patches listed below. They add the possibility to create map images from the server and the client with a list of options.

01/05 [mapimg] implement map image (patch #2018, In Progress)
02/05 [mapimg] map image support for the client (patch #2019, In Progress)
03/05 [mapimg] script to generate animated gifs from map images (patch #2020, In Progress)
04/05 [mapimg] add magickwand as another toolkit for the creation of map images (patch #2021, In Progress)
05/05 [mapimg] remove save_ppm and -P option (patch #1928, In Progress)

The patches are working at the moment. They add new features to freeciv. Thus, code is only added (save the last patch which removes the old code).

TODO:
- check all error messages; must they be translated?
- the colors are hardcoded. Is it possible to define them on the server and send them to the client (i.e. terrain and player colors in the ruleset)

need help:
- configure magic; I would like to have an option '--mapimg=[magickwand|sdl|gtk]' which activates the corresponding toolkit. Additional '--magickwand[=DIR]' could be needed. At the moment only magickwand and the default ppm toolkit are available.

Comments welcome!

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sat 02 Oct 2010 10:48:39 PM UTC, comment #17:

updated version:

  • working from the client (game => Save Map Image As ...)
  • options via Save => Options => Local Client => Map mage

(file #10572)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sat 02 Oct 2010 11:44:11 AM UTC, comment #16:

updated version:

  • only ppm files possible (no additional tools needed)
  • save mapimg definitions in savegame
  • show borders/cities below fow
  • prepare for using it on server and client
  • some cleanup

(file #10547)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sun 05 Sep 2010 05:31:17 PM UTC, comment #15:

> I meant borders under fog of war when both 'f' and 'b' options
> are enabled.


This is not possible at the moment. There are some problems to solve:

  • is it really needed? (or is it to much?)
  • it is only possible if the data for only one player is displayed.
  • at the moment all information of the server are used. For this the data in the struct player_tile must be analysed (limited knowledge of the player).
  • a rewrite of this code is needed (and planed!) to use it in the server and the client. I think there should be a interface in the client to save a map image on the client side.
Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sun 05 Sep 2010 05:13:44 PM UTC, comment #14:

I meant borders under fog of war when both 'f' and 'b' options are enabled.

pepeto <pepeto>
Project Member
Sun 05 Sep 2010 05:09:42 PM UTC, comment #13:

> It looks great. Is there a way to keep borders out of the
> vision?


yes; see the help

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sun 05 Sep 2010 05:05:11 PM UTC, comment #12:

It looks great. Is there a way to keep borders out of the vision?

pepeto <pepeto>
Project Member
Sun 05 Sep 2010 04:59:07 PM UTC, comment #11:

and a second example showing how the AI gives and remove vision ...

(file #10185)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sun 05 Sep 2010 03:20:25 PM UTC, comment #10:

add an example file

(file #10180)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sun 05 Sep 2010 03:17:30 PM UTC, comment #9:

an update after a long time ...

changes:

  • implement it as part of ./common
  • it will replace the creation of ppm files
  • addition engines using magicwand/gtk/sdl possible
  • can be called from the server (implemented), from the client (TODO) or as its own client (TODO)
  • needs the colour definitions in the ruleset (player colours, terrain colours, map overview colours)
  • can the file ./data/misc/colors.png be generated on the fly by the clients? Thus the colours for the players would be really independent. I think it should be possible for the gtk and sdl clients at least.

how to test it:

  • apply the patch
  • start the server
  • define an map image using 'mapimg define zoom=2'
  • start the game ...
  • use the script mapimg2anim.bin to generate an animated gif file

(file #10179)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sat 30 Jan 2010 07:44:52 PM UTC, comment #8:

time for a little status report:

I used the stub client (with some functions from the gtk client) for the new mapimg client. After the server is running it can be started as:

./civ.x --gui mapimg -a -s mattsys -n o1

It will autoconnect as player, drop the player status and switch to global observer. While the game is running it will create a map image for each turn. Most of this is hardcoded.

WIP:

  • a console similar to the one of the server for this client (for this console.(c|h) was moved to ./common); here some hints how this is / should working would be helpfull
  • remove the dependency on ImageMagick (MagickWand); use gtk instead (see mapimg.c)
  • some speedup of the image creation
  • cleanup of the code is needed
  • the ./civ script (see ./bootstrap/civ.in) does not allows to start different clients. My version (civ.x) uses a bad hack to allow the start of different clients. I will send this as a bug shortly.

(file #7846, file #7847)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sat 16 Jan 2010 10:33:45 PM UTC, comment #7:

The patch I posted here is working but after I had the time to read some additional code (see comment #5) I found some problems with this approach:

  • it needs a new dependency (MagickWand); using for example gtk one can also create image files (see gdk_pixbuf_save())
  • the images would be created by the server; this results in a mess with regard to some *.spec files. Should colors.tilespec be known by the client and the server? Does this definition depend on the tileset or not? Terrain color definitions in the tileset or in the ruleset?

I propose the following solution:

  • create a minimal (gtk) client which works as an observer (no gui!)
  • its purpose is to write an image of the map for each turn
  • it can be started from the server (using fork()) or as standalone binary which connects to the server as observer
  • it also can read a savefile and creates the map images
  • one can observe one player or the entire game

I will have to rework a lot of code for this. Before I start I would like to know how you think about this topic.

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Tue 12 Jan 2010 09:03:05 AM UTC, comment #6:

> This patch for generating a animated statistics from a game
> is exactly a feature I've been looking for, to generate
> post-game statistcs for http://www.freeciv.net/
>
> I'd like to use this functionality to create an animated .gif
> to be displayed on the website after the game is finished.
>
> Do you have any ideas, plans, or code, for how to trigger your scripts
> properly once a game has ended, so that the resulting animated .gif
> will be related to game statistics in a database?
>


At the moment the patch only creates map images. It can be configured to create different types of images (all players, only one player, ...). The animated gif (or mpg/avi/flv) is created by a script. If the server exists after the game ended you could call in a script like

# ./ser
# civstats2anim <options map animation 1>
# civstats2anim <options map animation 2>
# ...
# civstats2anim <options map animation n>

This way all animated gifs will be created after the server stops.

>
> I've found something similar here:
> http://munk.wtf.la:5560/game_records.php?game_id=11
> Are you the creator of this project also? If so, do you mind
> sharing the source code? Are all these statics automated?


The code for this statistics was created by munk. It is a php script available in the forum (http://forum.freeciv.org/viewtopic.php?t=6128). I created the patch after I read the script and ask myself: Why create a parser for the savegame if we have an excellent one (the freeciv server)?

The patch should be seen as a helper for the script. This way the map images as well as statistical data (as civscore.log file or *.csv files for each turn) can be created by the server and used as input for the postprocessing of the played game.

> I think we should cooperate on this functionality, since
> I'd like something very similar.
>
> On a related note, the Freeciv web client could be used to replay
> whole games, as well. This would involve setting up an observer
> player, which stores JSON in a database, which is then replayed
> to other players when they want a replay.


I'm not familiar with the freeciv web client but I think that it would be easier to create a save file for each turn and replay the game loading this files turn by turn and displaying the whole map. This is that the map images do (missing are the exact information about the units and cities).

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Mon 11 Jan 2010 09:59:07 PM UTC, comment #5:

The attached patch shows the current state of this patch. It does:

  • mapimg command (define, show, delete map image definitions & create map images)
  • it uses the ImageMagick C-binding Magickwand if available; fallback are ppm files
  • save / restore map image definitions in the savegame
  • bash script which creates animations from saved map image files (using convert from ImageMagick and ffmpeg)
  • create csv files with statistical data for each turn
  • save playercolor within civscore.log file
  • new binary to generate map images and csv files for savegames

TODO:

  • yesterday I found the files ./clients/colors_common.c and ./data/misc/colors.tilespec. The struct color and also the color definitions within the colors.tilespec file should be used. I plan to move the color definition for the terrains into terrain.ruleset as done within the patch (new capability needed?!) and split the remains of the file into playercolors.spec (with 32 colors from score.c) and gamecolor.spec (all the other color definitions).
  • Within ./client/overview_commen.c similar functions as in server/mapimg.c are defined. Perhaps some code can be reused.
  • The patch will be splitted into small parts.
  • Perhaps create versions for S2_1 and S2_2 (only the extra binary)?

Attached is an example animated gif image (on a full terrain map cities, units and borders are show for one player considering known tiles and fogofwar).

Comments welcome! (especially if I missed more files which code I have / will possible 'recreate')

(file #7660, file #7661)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Mon 11 Jan 2010 12:55:32 PM UTC, comment #4:

> Within colors_common.c it is said, that the number of player
> colors is limited to 14. Is this limitation still valid? I
> would like to use 30 different colors as defined in score.c?
> This way the colors would be different for each player.


After some more searching I looked into colors.spec and colors.png. Aha, these files have to be modified for 30 player colors ... ;-)

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Mon 11 Jan 2010 11:53:54 AM UTC, comment #3:

Going on yesterday (or today in the early morning), I found that the overview has similar definitions like I have writen. I will check if I can (re)use some of the code.

> But I guess that the .tilespec will have to be dropped in this
> case.


That do you mean? Is an other file extension needed?

Furthermore, colors.tilespec defines colors for each terrain (hard coded names). As the definition of terrains is variable (number, names, ...), these coors should be within the terrain.ruleset file (as something like 'color = 200, 200, 200' for each terrain). This would require an new version of this ruleset file.

Within colors_common.c it is said, that the number of player colors is limited to 14. Is this limitation still valid? I would like to use 30 different colors as defined in score.c? This way the colors would be different for each player.

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sun 10 Jan 2010 11:20:09 PM UTC, comment #2:

It sounds like a good idea for me. But I guess that the .tilespec will have to be dropped in this case.

pepeto <pepeto>
Project Member
Sun 10 Jan 2010 11:10:31 PM UTC, comment #1:

I found today that player colors as well as terrain colors are defined within ./data/misc/colors.tilespec. They are read by functions within ./client/colors_common.(c|h). Would it be OK to use these colors for the creation of an map image?

This would mean, that colors_common.(c|h) are moved into ./common and that the server would use these functions to read the colors from colors.tilespec to create the images.

The basic question is, if the colors.tilespec file will be available for the server like the ruleset files?

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.
Sat 19 Dec 2009 11:23:08 PM UTC, original submission:

If freeciv is played statistics are interesting, most of all as images. The attached patch adds a new binary called civstats which reads a savegame and creates a ppm images with

  • the map (ocean / land)
  • cities and units
  • border

The code was tested for all topologies available. It will be extended to

  • return a ppm image of the map (terrain)
  • return a txt file with statistics (can be easily interpreted by a script to generate graphs)

using a small bash script and ppmtogif and convert animated gif images can be created.

Matthias Pfafferodt <syntron>
Project MemberIn charge of this item.

 

(Note: upload size limit is set to 1024 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #10847:  mapimg-20101020.diff.bz2 added by syntron (68kB - application/x-bzip2)
file #10766:  mapimg.diff.bz2 added by syntron (63kB - application/x-bzip2)
file #10749:  mapimg.diff.bz2 added by syntron (51kB - application/x-bzip2)
file #10572:  mapimg.diff added by syntron (130kB - text/x-diff)
file #7846:  mapimg.patch added by syntron (279kB - text/x-diff)
file #7847:  civ.x added by syntron (2kB - application/x-shellscript)
file #7660:  statistic.patch.bz2 added by syntron (28kB - application/x-bzip2)
file #7526:  civstats.patch added by syntron (39kB - text/x-diff)
file #7527:  civstats2gif.bin added by syntron (528B - application/x-shellscript)

 

Digest:
   bug dependencies, patch dependencies.

Digest:
   patch dependencies.

 

Carbon-Copy List
  • -unavailable- added by cazfi (Updated the item)
  • -unavailable- added by jtn (Posted a comment)
  • -unavailable- added by pepeto (Updated the item)
  • -unavailable- added by syntron (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 25 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 13 Aug 2014 12:12:22 AM UTCcazfiStatusIn Progress=>Done
      Open/ClosedOpen=>Closed
    Wed 13 Aug 2014 12:12:09 AM UTCcazfiDependenciesRemoved dependancy to bugs #16887=>-
    Wed 13 Aug 2014 12:12:04 AM UTCcazfiDependenciesRemoved dependancy to bugs #16864=>-
    Mon 25 Mar 2013 08:33:49 AM UTCcazfiDependencies-=>Depends on bugs #19168
    Thu 03 Mar 2011 12:32:09 AM UTCjtnDependencies-=>Depends on bugs #17826
    Wed 12 Jan 2011 11:22:55 AM UTCsyntronDependencies-=>Depends on patch #2313
    Thu 28 Oct 2010 01:18:55 PM UTCsyntronPlanned Release2.3.0=>2.4.0
    Wed 20 Oct 2010 11:48:01 AM UTCsyntronAttached File-=>Added mapimg-20101020.diff.bz2, #10847
    Mon 18 Oct 2010 03:17:29 PM UTCsyntronDependencies-=>Depends on patch #2072
    Mon 18 Oct 2010 03:11:27 PM UTCsyntronDependencies-=>Depends on bugs #16887
    Thu 14 Oct 2010 10:54:55 AM UTCsyntronDependencies-=>Depends on patch #2069
    Wed 13 Oct 2010 06:03:13 PM UTCsyntronAttached File-=>Added mapimg.diff.bz2, #10766
    Wed 13 Oct 2010 05:50:54 PM UTCsyntronDependencies-=>Depends on patch #2066
    Wed 13 Oct 2010 04:37:54 PM UTCsyntronDependencies-=>Depends on bugs #16864
    Tue 12 Oct 2010 01:14:52 PM UTCsyntronAttached File-=>Added mapimg.diff.bz2, #10749
    Tue 12 Oct 2010 01:08:12 PM UTCsyntronDependencies-=>Depends on patch #2063
      Dependencies-=>Depends on patch #2062
      Dependencies-=>Depends on patch #2061
      Dependencies-=>Depends on patch #2060
      Dependencies-=>Depends on patch #2059
      Dependencies-=>Depends on patch #2058
      Dependencies-=>Depends on patch #2057
    Sun 03 Oct 2010 04:50:11 PM UTCsyntronSummarynew code to generate stats (graphics)=>[Metaticket] new code to generate stats (graphics)
    Sun 03 Oct 2010 04:43:10 PM UTCsyntronDependencies-=>Depends on patch #2021
    Show feedback again

    Back to the top


    Powered by Savane 3.1-cleanup