bugFreeciv - Bugs: bug #17962, Inability to generate/load huge...

 
 
Show feedback again

bug #17962: Inability to generate/load huge maps on Windows

Submitted by:  None
Submitted on:  Tue 29 Mar 2011 10:34:45 AM UTC  
 
Category: client-gtk-2.0Severity: 4 - Important
Priority: 5 - NormalStatus: Fixed
Assigned to: Jacob Nevins <jtn>Originator Email: -unavailable-
Open/Closed: ClosedRelease: 2.3.0-beta3
Operating System: Microsoft WindowsPlanned Release: 2.3.0,2.4.0

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 14 Jul 2011 10:17:01 PM UTC, comment #20:

Thanks to some Windows builds provided by cazfi, I'm reasonably convinced that the patch has indeed fixed the problems described in this bug.

Gory details for the record only:

Official 2.3.0-beta4 build from Sourceforge (control 1; expected result: fail):

  • Reproduced comment #0: start standalone server; "set size 80"; "set minplayers 1"; "start" => server silently dies (last output is "2: Creating a map of size 348 x 232 = 80736 tiles (80000 requested).", even with "-d 3")
  • Reproduced GM1530's huge-scenario woes (comment #4):
    • The one remaining map linked here (earth-500x250-v1.0.sav.gz) silently kills a standalone server
    • A random selection of maps marked as troublesome in the forum topic do indeed kill the server
      • europe-350x350-v1.1.sav.gz
      • northamerica-350x350-v1.1.sav.gz
    • A selection of maps marked as OK are indeed OK
      • europe-332x264-v1.8.sav.gz
      • europe-350x350-v1.0.sav.gz
      • northamerica-350x350-v1.0.sav.gz

cazfi's S2_3 r19844 snapshot + winstack-0.6.4 (control 2; expected result: fail):

  • standalone server with "/size 80": died more noisily (and Googling suggests that "Exception Code: c00000fd" is indeed a stack overflow):

cazfi's S2_3 r19947 snapshot + winstack-0.6.5.103 (under test; expected result: pass):

  • "/set size 80": works fine
  • loading all of the scenarios mentioned works fine
Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Wed 13 Jul 2011 11:42:11 PM UTC, comment #19:

OK, I'm less worried now; I think the observed non-determinism isn't caused by this patch; see bug #18346. (Plus after more runs it's not perfectly correlated with the presence of the patch.)

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Wed 13 Jul 2011 10:18:38 PM UTC, comment #18:

Tested S2_3 before and after this patch with an autogame.

Good news: the autogame with the patch ran to completion without anything obviously bad happening.

Bad news: this seems to have perturbed the outcome of autogames. In fact it's different from the very start: the order in which start positions are assigned to players is different (although the nations/leaders are the same).
The change seems to be deterministic, at least: two autogames without the patch gave similar-looking results up to the point I stopped them, and the same with the patch.
This doesn't actually break anything, but it's worrying, as I don't think this patch should have perturbed anything like this -- even the extensive change to assign_continent_flood() shouldn't have changed the continent numbers, I think.

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Wed 13 Jul 2011 08:59:01 PM UTC, comment #17:

Done that.

Based on nothing more than Marko's diagnosis in comment #12, I'm going to mark this as fixed, in an attempt to gather evidence to the contrary.

If anyone able to make Windows builds can actually verify that the original problem has gone away before the weekend, that would be great. Or just make a Windows build available containing this fix and I'll compare it to one without myself (if I find time).

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Wed 13 Jul 2011 08:56:23 PM UTC, SVN revision 19941:

Changes to avoid stack overflow with large maps:
- many instances of allocation of map-sized arrays on the stack have been
replaced with heap allocation
- assign_continent_flood() no longer uses recursion

Patch by akfaew@gna and Matthias Pfafferodt (syntron@gna) with a few tweaks
by me (originally part of a patch in gna bug #18087).

See gna bug #17962.

(Browse SVN revision 19941)

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Wed 13 Jul 2011 08:48:15 PM UTC, SVN revision 19940:

Changes to avoid stack overflow with large maps:
- many instances of allocation of map-sized arrays on the stack have been
replaced with heap allocation
- assign_continent_flood() no longer uses recursion

Patch by akfaew@gna and Matthias Pfafferodt (syntron@gna) with a few tweaks
by me (originally part of a patch in gna bug #18087).

See gna bug #17962.

(Browse SVN revision 19940)

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Mon 11 Jul 2011 10:59:42 PM UTC, comment #14:

I intend to apply file #13531 (from bug #18087) to try to deal with this before 2.3.0-RC1 is released.

Since there isn't much time before RC1, pre-commit testing would be appreciated (especially by Windows users).

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Tue 21 Jun 2011 09:22:36 PM UTC, comment #13:

Aha. Bug #18087 also has assign_continent_flood() as a problem.

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Tue 21 Jun 2011 09:18:12 PM UTC, comment #12:

Map creation problem reproduced in Win7 using only server (trying to launch autogame).

I even got some kind of backtrace: segfault in map_pos_to_index(). Before that backtrace is filled with assign_continent_flood() - and by filled I mean that I stopped listing them after 15000.

So it seems like unlimited recursion in assign_continent_flood() eventually causing stack overflow.

Marko Lindqvist <cazfi>
Project Administrator
Thu 09 Jun 2011 08:04:33 AM UTC, comment #11:

>> Server should be as fast in Windows as in Linux, but Windows gtk is definitely slower.


I think that the problem is Windows gtk.

J. M. Gorbach <gorb>
Tue 07 Jun 2011 12:28:06 PM UTC, comment #10:

...or network (connection between server and client) buffers overflowing with such a massive amount of data. That could happen if for some reason server is sending data faster than client accepts. Server should be as fast in Windows as in Linux, but Windows gtk is definitely slower.

Marko Lindqvist <cazfi>
Project Administrator
Tue 07 Jun 2011 12:22:34 PM UTC, comment #9:

What kind of hardware you have? I just wonder if this could be simply problem of map generation taking so long that connection between client and server timeouts? Or server running out of memory for such a massive map?

Marko Lindqvist <cazfi>
Project Administrator
Wed 06 Apr 2011 08:00:56 PM UTC, comment #8:

could you test, if the server of the windows build prints some usefull output before it exits (i.e. start the sever seperate)

Anonymous
Wed 06 Apr 2011 12:51:33 PM UTC, comment #7:

The same problem with 2.3.0-beta4 on a Windows Vista.

J. M. Gorbach <gorb>
Sun 03 Apr 2011 06:37:23 PM UTC, comment #6:

I did load the four scenarios without problem with S2_3 on an Ubuntu 10.10 32bit.

It must be a problem with windows binaries...

Jordi Negrevernis i Font <jorneg>
Project Member
Sun 03 Apr 2011 05:59:51 PM UTC, comment #5:

(I meant "user GM1530 on the forum [...]")

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Sun 03 Apr 2011 05:53:06 PM UTC, comment #4:

Another data point:

User GM1350 on the forum, who's put together a number of large map scenarios, reports in email that they cannot load some of them in the Windows version of 2.3.0-beta3, but they can load others (see thread). I think the examples mentioned were:

GM1350 writes: "If I use my maps (ex. Earth) and I select this map in ''Scenario Game'' (on Windows Vista).. I see this:
"But freeciv-server.rpt is white!"

Which seems to indicate that the server has crashed.

I haven't tried to work out in detail what's different between the reported working and non-working ones. The non-working ones tend to have a larger number of tiles, but there isn't a simple threshold in number of tiles that says whether or not it'll work.

However, I have no trouble loading any of these scenarios on Ubuntu Linux 10.04.2 on x86_64 (64-bit), either in a client-spawned server or in a standalone server (and I don't see any errors or warnings in the latter), either with 2.3.0-beta3 or the head of S2_3.

Similarly, I have no trouble generating maps where size=90.

So it looks like we're dealing with a Windows-specific server crash, which is a shame as I don't really know how to get good post-mortem diagnostics for Windows.

I guess it's also possible that it's an issue when Freeciv is compiled for 32-bit architectures -- some quantity that needs to be 64-bit is not? Can someone confirm whether the above scenarios work when compiled for some 32-bit Unix platform?

Possibly running valgrind on Linux will show something up; I haven't tried it yet.

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Thu 31 Mar 2011 09:15:10 PM UTC, comment #3:

(I assume there are two different anonymous people having a conversation here?)

To the original reporter: it might help if we can see your settings. Can you type "/show all" in the pregame window and paste the resulting output here?

Also, can you try with a standalone server (rather than a client-spawned one)? If you can reproduce this that way, you should be able to see any error messages the server produces before it dies, which would be useful to know.

Jacob Nevins <jtn>
Project AdministratorIn charge of this item.
Thu 31 Mar 2011 08:55:25 AM UTC, comment #2:

I have tried it many times with two different PCs, first running on Windows 7 and the second one on Windows XP SP3 and many different settings but with no success

Anonymous
Tue 29 Mar 2011 08:52:23 PM UTC, comment #1:

I am able to generate maps up to size 128 with 2.3.0-beta

although it does it errors about being unable to create starting positions (I used 126 players with 120k tiles), it doesn't lose connection with the server, and the game starts normally with a huge map.

I'm using Ubuntu 10.10, not windows, maybe this is a windows issue

Anonymous
Tue 29 Mar 2011 10:34:45 AM UTC, original submission:

Program is not able to generate maps of size above 70-80 thousand tiles (exact number is a bit varying). After chosing "Start" it just returns to the main screen with error message "Lost connection to server (read error)!"

Anonymous

 

(Note: upload size limit is set to 1024 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by cazfi (Posted a comment)
  • -unavailable- added by gorb (Posted a comment)
  • -unavailable- added by jorneg (Posted a comment)
  • -unavailable- added by jtn (Posted a comment)
  • -unavailable- added by None (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 6 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Wed 13 Jul 2011 08:59:01 PM UTCjtnStatusReady For Test=>Fixed
      Open/ClosedOpen=>Closed
    Mon 11 Jul 2011 10:59:59 PM UTCjtnStatusNone=>Ready For Test
      Assigned toNone=>jtn
    Wed 15 Jun 2011 09:59:47 PM UTCjtnSummaryInability to generate huge maps=>Inability to generate/load huge maps on Windows
    Sun 03 Apr 2011 05:53:06 PM UTCjtnPlanned Release=>2.3.0,2.4.0
    Show feedback again

    Back to the top


    Powered by Savane 3.1-cleanup