Wednesday, April 27, 2016

One year after

I started developing Squadron Benchmark one year ago. So, between bug corrections, I'll take the time to discuss what I've learned and what to expect for the following year.

Evolution of the code

The code is still buggy, but nothing compared to the first 0.5 version online one year ago :-) 
I wish to thank all  of you who report bugs, provide encouragements and donate for Squadron Benchmark. The code would not be in its current state without you ! Testing this kind of webapp automatically is nightmarish, and developing with no test goes nowhere, so actually when you report bugs, you contribute to half the development, really.

The app, in term of code, uses javascript, HTML5, CSS3 and SVG. SVG+javascript is rather convenient for displaying moving units, computing intersections and distances, the display of the game area is rather stable. CSS3 is powerful enough for some fun 3D effects (maybe too many in squad creation...).

Now for javascript development. After one year of development, the code has 15000 lines of javascript, with 7800 lines just to describe the specificities of 180 pilots and 190 upgrades. The rest is the game engine. Roughly speaking, each pilot and upgrade requires, on average, 15 lines of code to describe its special effects in term of game mechanics. During this past year, I counted ~100 new cards published by FFG, including pilots and upgrades. If FFG publishes the same number of new units the year to come, this means 1500 new lines of code, assuming there is no new game mechanics. That should be sustainable !

 The game engine by itself still need some improvement, some rules are not fully implemented, the X-Wing rule FAQ does not yet apply completely. This is one of the main goals of this second year of development.

Artificial intelligence

The AI is very simple. There are 340 lines of code dedicated to the AI, so no miracle ! This was not on top of my priorities, bug correction first, but critical bugs tend to be solved, so it's time for AI. One possibility is to let all players customize the AI of the squads. There is still some effort here before it can be customized, and I need to implement higher level functions.
This is one of the main objectives of the forthcoming developments.

All the successfully completed combats are automatically registered in Google spreadsheets. You can replay any of them through battle log (the black book icon).  There are actually 7 spreadsheets with 2000 combats each, so ~14000 combats are available on line. I think it's an unused feature still, probably because you have to select or create a squad before checking its battle log. It's not possible so far to directly request the database for squads.  I'm open to any suggestion anyway on how to use this, that's the only way to share information between players so far.
I use this database to compute some stats however, such as the "quality" of the AI. For instance, the average score, computed every 200 combats between player/AI, is shown in the following figure.
So one vertex corresponds to 200 combats, and the y-axis shows the % of total points won by the AI. The x-axis corresponds to time (from Oct. to Apr.)
Here the AI makes in average 25% of the score. So for a 200 point combat, the average score would be 50 for the AI, 150 for the human player. This is an average, most of the time, the AI as a null score but from time to time, it manages to win.

Multiplayer game

Squadron benchmark includes most of the rules, including rules of the upgrades/pilots. As this is only a one player simulator, it's a complementary tool for the real table top game, for training and testing. As it was said in other places, it has not the look and feel of a miniature game of course, and playing for real against a real human is way better. Making the app for two player games, it could be seen as a computer alternative to the real game. I'm not going to do it without the explicit agreement of FFG.

The Dark Side of Squadron Benchmark

The source code of Squadron Benchmark is available on line, on github. However, it's developed using vanilla javascript and css, there is no developer documentation, no comment, no test. As my time dedicated to this is limited, I prefer to code rather than document or coordinate the dev. I do it for the fun, for the challenges, and for trying new ideas and new techs, I don't care about a clean project management. Don't expect too much in this direction therefore ! :-)

I'll document this anyway, at some point. As I'm not especially competent for the development of AIs, this is a good way to contribute if you wish :-) (well, as soon as a minimum set of functionalities for the AI is ready to be used anyway). 
In particular, if by chance someone is motivated by deep learning and willing to adapt/use ConvNetJS for Squadron Benchmark, I'll gladly help ! I don't know if the AI can learn by itself how to play and find interesting strategies, or even if ConvNetJS can scale to the complexity of X-Wing miniature game rules, but it's worth a try and should be fun !


Cheers !

54 comments:

  1. Thanks again for doing this! I can't imagine having to code all of this myself.
    Major bug report: It's not major in the sense that it crashes, but Dash Rendar cannot use actions on asteroids. It makes flying him really hard compared to real life.

    ReplyDelete
    Replies
    1. Thanks ! And corrected, now he cannot fire but he can make actions. Cheers !

      Delete
  2. Great work! I just ran into a bug where the game stalls when Dengar's ability interrupts between two Twin Laser Turret shots.

    ReplyDelete
    Replies
    1. Thanks ! For Dengar, should'nt it be the case ?
      Cheers !

      Delete
    2. Dengar should interrupt, yes, but the program should not be unable to proceed when that happens. :-)

      Delete
  3. For some reason, this squad prevents the program from running:
    Prototype Pilot + Chardaan Refit + Autothrusters
    Blue Squadron Pilot + Fire-Control System
    Blue Squadron Pilot + Fire-Control System
    Wedge Antilles + Predator + BB-8 + Integrated Astromech

    ReplyDelete
    Replies
    1. It works for me (just copy/pasted your list), when played by me or the AI. What are the symptoms ?

      Delete
    2. It's working for me now. Thanks!

      Delete
    3. I'm not certain, but it seems that Dengar is triggering his pilot ability even when the enemy is outside of his firing arc.

      Delete
    4. I've tested a Dengar / Han Solo combat, Dengar is not retaliating when Han is not in his firing arc. Dengar had some special upgrade ? Cheers !

      Delete
  4. Hey man great job on this tool. This has really come a long way in the 4 short months that I have been using this.

    I have noticed a bug similar to David R, where when I enter a squad there is a long delay while the cursor flashes between the waiting icon and the normal pointer, before it loads the squad. I get the same thing when I try to load the benchmark and test the squad, the only notable difference is the benchmark won't actually load. This occurs without regard to my squad setup.

    I also noticed that Whisper/Phantoms when controlled by AI wont decloak until right before they move instead of at the beginning of the activation phase. This seems to only affect the AI, a player still must decloak at the beginning of the activation phase, as normal.

    One last bug I noticed is it seems when the Inquisitor is attacking at range 3, the defender is still getting their range 3 defense die, instead of treating the range as range 1.

    ReplyDelete
    Replies
    1. Thanks ! For the loading time, something goes wrong here. Best to load again the page (flushing the cache if you can [pressing shift with FF or Chrome]).
      For Whisper/Phantom, decloak was not showing when it occurred (sound was playing). Corrected !
      For Inquisitor, corrected, I had forgotten to add the effect on defense too. Cheers !

      Delete
    2. Awesome, works like a charm!

      Delete
  5. Seems to be a bug in the squad building. When I go to click "Next" it spins between the load and pointer icon and locks up the browser.

    ReplyDelete
    Replies
    1. Hmmm, this is a volatile bug, between two updates. Just reload the page or close the browser and start again.

      Delete
  6. Since today I experience a bug - as soon as the computer ships fly over an obstacle the game comes to a stop. The game don´t proceedes onward to the other ships after that.
    I tested this with 2 browsers (mozilla and IE). I am the only one with this bug?

    ReplyDelete
  7. Same here with google chrome. As soon as a computer ship overlaps an asteroid the game stops.

    ReplyDelete
  8. I'm having the same issue (using safari on an iPad).

    Great work though, I love using this to test out squads.

    ReplyDelete
    Replies
    1. Yep, I'm having the same issue using Firefox 45 on Mac.

      It also hangs sometimes when certain ships are supposed to fire. I think it was the Inquisitor.

      Additionally, the asteroids seem overly large now. Is that by design?

      Delete
    2. Ok for obstacle collision. For asteroids, yes they seem to have grown since the addition of debris...Strange.

      Delete
  9. game hangs after a ship flies over an asteroid
    log says:
    [Blue Squadron Pilot] unit or template overlaps obstacle: no action
    then the game stops.

    ReplyDelete
  10. There are times when one ship fires using special circumstances (such as a crit not being able to be canceled with Ten Nunb) where after the attack hits damage is not dealt and none of the other ships are given a chance to fire with the only option being to hit 'next' to get to the next planning phase. Thank you so much for your work on this project!

    ReplyDelete
    Replies
    1. Thanks ! Do you remember the unit that last fired, (with all its upgrades), so that I can reproduce the bug ?
      Cheers !

      Delete
    2. I've been getting similar issues with these ships firing:
      Corran Horn + Draw Their Fire + R2-D2 + Hull Upgrade
      and
      Redline + Guidance Chips + Extra Munitions + Concussion Missiles + Fire Control System

      Delete
    3. Also had it with
      Darth Vader + TIE/x1 + Advanced Targeting Computer + Predator + Twin Ion Engine mk 2.

      Delete
  11. I want to thank the X Wing Benchmark team for the excellent work . Very intuitive and useful for training with newly created builds. But I do not know the appropriate place to report bugs , so I'll comment here.
    1- After the last update, when the ship overlap or collide with the asteroid , the game completely lock , you can not assign any more maneuver and not jump to the next step .

    2. The ability Bossk does not work , you can not change 1 hit critical for normal 2 hits.

    3. After using the Illicit , feedback array, no one can attack the combat phase has to be skipped.

    I am very grateful for the great training program , I found it better than the Vassal ;-) and I will make my donation .

    ReplyDelete
    Replies
    1. Thanks a lot ! Feel free to report bugs here.
      1- corrected !
      2- it works (I think), but is fully automatic: if there are enough shields to absorb the critical, or if the enemy unit has less than 2 hulls remaining, then the capacity of Bossk is triggered. Otherwise, a message like "..keeping critical" is displayed.
      3- Its maybe not very clear, but just before combat, the units at range 1 of the feedback array are flashing. Select one of the to activate the array (or the unit carrying the array to cancel). The console should display a message "select unit (or self to cancel)[feedback array]".
      Cheers !

      Delete
  12. Ya, after the latest update it seems that if a ship hits an obstacle, it will stop the game. You can hit "ESC" to continue the game, but the ships that haven't moved yet don't get a chance to move - it goes right into the combat phase.

    ReplyDelete
  13. Also, the asteroids seems way too big...

    ReplyDelete
    Replies
    1. Yes, the scale factor for the debris cloud was applied to the asteroids. Corrected.

      Delete
  14. This comment has been removed by the author.

    ReplyDelete
  15. Thanks Denis, for keeping this up!

    Here's another bug, this time in Replay. The Replay stops abruptly, I think due to Jan Ors.

    Here is a tinyurl to the Replay:
    http://tinyurl.com/jkn6t6s

    ReplyDelete
    Replies
    1. It works now ! Thanks for the tinyurl :-)

      Delete
  16. Hey Denis

    good job with the recent corrections. Thanks. Thumbs up.

    I encountered two more:
    - When Carnor is eliminated his special continues to work measured from his last position (cannot use Focus).
    -Hobbie Klivian and Targeting Astromech: You should be able to let Hobbie taking an action after clearing the stress after K-Turn using the Mech.

    Thanks again for your effort and dedication!

    ReplyDelete
    Replies
    1. Thanks a lot !
      For Carnor (and others, with effects on other units), it's corrected.
      For Hobbie and Targeting Astromech, the Mech clears the stress but there is no action indeed. Timing issue here, I haven't yet found a solution...

      Delete
  17. Hey Denis

    good job with the recent corrections. Thanks. Thumbs up.

    I encountered two more:
    - When Carnor is eliminated his special continues to work measured from his last position (cannot use Focus).
    -Hobbie Klivian and Targeting Astromech: You should be able to let Hobbie taking an action after clearing the stress after K-Turn using the Mech.

    Thanks again for your effort and dedication!

    ReplyDelete
  18. Ran into an issue where Howlrunner's ability was still available to other ships in the squadron even after Howlrunner was destroyed.
    Howlrunner + Juke + Twin Ion Engine mk 2
    Black Squadron Pilot + Crack Shot x4
    Gamma Squadron Pilot + Twin Ion Engine mk 2

    ReplyDelete
    Replies
    1. There was a problem with Juke (corrected); Howlrunner effect checks if Howlrunner is dead, just in case.

      Delete
  19. Thank you. This has been great.

    One bug report: Sound no longer works after toggling it. How can I reset this?

    ReplyDelete
    Replies
    1. In previous versions of the app, you could toggle the sound (on or off). Now you can change the volume of the sound. Just reload the page, you should see version 0.8.8 (at least). Cheers !

      Delete
    2. Thanks. Pressing the 'reload' button on the app solved the issue. Though today I notice that asteroids and Team 2 aren't loading properly at startup. Team 2 is stacked facing up in the top left corner.

      Delete
    3. Ah, depends on what is in Team2. If there's a TIE/D, this is solved. Otherwise, tell me is your squad list :-)

      Delete
  20. Thank you so much for your work! I've been using this a ton over the past few weeks.

    One bug report: Recently I noticed that any list with at least one TIE/D title in it stacks all ships in that squad on the top left corner of the map at the beginning of setup, does not allow you to select asteroids, and does not allow you to move past the first planning phase. I am using Google Chrome version 50.0.2661.102 (64-bit) on Mac OS X version 10.11.5. Using TIE/Ds was fine a couple of weeks ago but I can't give an exact date.

    Thank you again!

    ReplyDelete
  21. I'm having the same issue as the user "Unkown May 20, 2016 at 7:16 AM" I'm using Firefox 46.0.1 and Mac OSX 10.10.5.

    Thanks!

    ReplyDelete
  22. Same problem here on Chrome and Firefox latest versions.

    Thanks for all your work with this system - it's really amazing!

    ReplyDelete
  23. Hi,
    this is a great and very useful tool. Are you planning to add epic ships in a future version?

    ReplyDelete
    Replies
    1. Thanks ! For epic, why not, but it's not on my priority list though :-(
      Cheers !

      Delete
  24. Hi there!

    I cannot find the code on github... what am I missing?

    Thanks

    ReplyDelete
  25. https://github.com/xws-bench/bench
    Cheers !

    ReplyDelete
  26. Drea Renthal + Twin Laser Turret + R4-B11
    Just makes the whole combat phase have to be skipped mid-way thru her 1st tlt shot. Not sure why, the UI for the 'Done' button just doesn't display.

    ReplyDelete