Mercurial/Git hosting found with Trac support!

[Technomage]

While looking for a host to upload my local mercurial repository I ran across Assembla (www.assembla.com). Barra and I had previously evaluated assembla and decided that the bug tracking system it used wasn't sophisticated enough at the time. However, it appears as though assembla now supports both mercurial and git with Trac integration!

I've been playing around with their free hosting option and was able to convert our svn repository to mercurial without too many issues. You can see the results here: http://hg.assembla.com//technopolitica/graph/839/?revcount=480. Mercurial tends to be conservative when deciding when to save branches, so there are a ton of old branches listed that don't really serve a purpose any more, but mercurial does provide some options to ignore or convert some branches and I think that issue is easily solved.

Their Trac hosting is a bit more limited than our current Trac hosting by Codesion, but Assembla provides other well-integrated tools that more than make up for that. Actually, after looking over Assembla in detail I think that their default bug tacker is much more user friendly than Trac even if it doesn't have all of the bells and whistles. In addition, Assembla is geared towards agile development processes and has the tools to support sprints and progress reporting that we currently use (hackish) Trac plugins for.

Assembla really looks ideal in terms of a distributed VCS for us. They appear to offer free hosting to open-source projects with unlimited developers and up to 2 GB of server space. However, I do have a few concerns that need to be addressed before we consider Assembla for our host:

• Although Assembla claims to provide free hosting for open-source projects, it was somewhat unclear if there will be restrictions on a free account;
• It was also unclear whether the 2 GB limit was a hard cap, and whether the repository size counts towards that limit;
• Although Assembla provides a Trac ticket database import feature our current Trac host, Codesion, has purposefully disabled exports of our database so that they can charge $10/month for backups (a rather dickish move on their part IMO);

I will have to email Assembla for clarification on these points, and also see if Codesion would be willing to make a one-time dump of our Trac database without charge. If Codesion is unwilling to do so, I can dump all of our tickets into a csv file and import them from there, but it is significantly more complicated and would require manually formatting the csv file into a form that Assembla can import.

[mvBarracuda]

Sounds good to me Technomage :-) Thanks for setting things up.

Are you sure about Trac integration for git&hg? Last time I checked out Assembla for PARPG (back in January AFAIR), they did only offer Trac integration for SVN, while Git (they didn't offer hg support at all back then) came with an alternative custom issue tracker software that lacked a lot of Trac's functionality.

[Technomage]

Yup, they do indeed have Trac support for svn, git and hg. I was just as surprised as you were to see that Assembla had implemented Trac support for git and hg. However, their version of Trac is really just a front-end to Assembla's custom bug tracker AFAICT, which at least to my eye has been significantly improved since we last looked at it. The query features of the Assembla bug tracker in particular seem much intuitive and user friendly than Trac.

Assembla seems to follow the unix motto of "one program per function" in the sense that they provide a range of modular, semi-independent tools. The type of repository is abstracted and hidden from the rest of the components, so I suspect that is why they were able to get git and hg Trac integration up and running so quickly.

In addition to a bug tracker and repository hosting which supports bug tracker commit hooks like we use with Trac, Assembla also provides a built-in wiki, forum-like messaging software, irc-like instant messaging server, and a configurable build server for use in continuous integration. Its actually quite an impressive array of tools - we could conceivably migrate our entire web infrastructure to Assembla if we wanted.

However, I do worry about the fact that Assembla seems to be geared towards commercial products - I don't want to get sucked into their free hosting only to discover later that we'll need to pay a monthly fee to use the tools we want and need. This apparently did happen with Codesion and yet they still support us, so I might just be overly paranoid.

[Technomage]

After a bit of work and lots of patching, I managed to successfully convert our svn repository to a mercurial repository with a relatively clean history. You can view the results at our Assembla page at www.assembla.com/spaces/parpg under the "Source/Hg" tab.

There are several things to note about the converted mercurial repository:

First, I pruned out all of the old branches in svn except the character_customization sprint branch. These branches were making it impossible to create a clean history due to the non-standard way we structure our branches, and weren't really adding much in terms of history anyway.

Second, I manually merged the character_customization branch into the default mercurial branch and threw away a couple of revisions on the character_customization branch that were mistakenly applied to the branch instead of trunk. For some reason the character_customization branch was not merged back to trunk in svn - rather, trunk was patched. In order to maintain a clean history in mercurial I decided to do a manual merge instead, preserving as much of the original history as possible there.

Third, I pruned out all references to the media folder in older revisions to save space. The media folder, which now resides in a separate branch, was adding ~300 MB to the mercurial repository which came in at about 600 MB total, which was simply unacceptable for a DVCS. Without the media references, the total size of the repository dropped to under 300 MB which is much more reasonable (but still quite high). As such, older revisions which relied upon the media folder may not function correctly (unlikely, but possible).

The media svn branch should probably reside as a separate mercurial repository so that developers don't have to clone the media branch with the rest of the code.

The mercurial repository is still quite large for a DVCS, and we may need to address that issue. We don't want clones to take hours over the 'net, after all. I'm not quite sure how to deal with the large size - the objects folder with the blender renderings are taking up the bulk of that space, with the music being a close second. Its possible that we may want to remove all of the blender renderings and replace them with the actual blender project files, and have the renderings generated as part of the build process. However, this would make blender a build dependency and make source builds quite a bit more complex.

[mvBarracuda]

Heya Technomage, thanks for setting everything up!

For some reason the character_customization branch was not merged back to trunk in svn - rather, trunk was patched.

Would that make any further changes to SVN trunk necessary? Or can we simply continue with the current SVN trunk as it contains all the changes from the last sprint branch?

Concerning branches/media: I think there are two options in this case:
1. Set up a seperate media hg/git repository where the media files (raw assets) reside.
2. Continue to use SVN for the media files. We don't really benefit from DVCS features for these specific files. The obvious drawback would be that we would be using two different kind of version control systems in PARPG in this case though.

Concerning the trunk/game/objects folder: rendering the assets in there on the fly on the target system is not an option to be honest. There are a couple of reasons:
1. I don't want to introduce Blender as an dependency to just test the game.
2. It wouldn't be only Blender, but we would even have to make 3dsMax & XSI|Softimage a dependency as not all models in there are available as Blender files. E.g. all the (N)PC animations have been done in XSI and aren't availabe as Blender files so we can't render them on the target system.
3. Even worse: some renders don't come with any raw asset versions at all. This means we can't reproduce these objects from source. While this shouldn't happen in open source project (missing raw assets), it's unfortunately the case. Some artists showed up, sent in renders we liked and suddenly vanished before sending in the 3d models and textures for these renders. In these cases we'll have to leave the rendered versions in the repository or replace these assets with new versions.
4. There are some objects in there that are based on 2d art instead of 3d models, e.g. the bottle that lies on the ground. We would have to handle 2d & 3d objects seperately; I don't think we should do that as it makes things even more complicated.

IMHO all the objects should stay inside our VCS repository for the sake of easy access and uncomplicated way of setting a test environment for new devs up.

[Technomage]

That sounds like a good assessment. Unfortunately I haven't been able to figure out a good solution to the repository size issue. Apparently mercurial is working on support for shallow clones (http://mercurial.selenic.com/wiki/ShallowClone), so that might be a viable solution for minimizing network traffic once it is implemented.

I found some more information about the free hosting offered by Assembla: http://offers.assembla.com/free-project-hosting. In summary, Assembla offers free hosting for public and open-source projects with an unlimited number of users and 2GB of space (AFAICT repository size does NOT count towards this limit, which is a good thing) and the majority of their tools are made available for free accounts. Perhaps most importantly, they guarantee 1 year of service at the terms you sign up for regardless of whether they discontinue that plan. This means two things: 1) we have 1 year of free service no matter what, 2) we ONLY have 1 year of free service (probably). This is only somewhat comforting, but I suspect that its the best terms we're likely to get these days.

[Beliar]

Not sure how unkown-horizons handles their gfx files -  and the size of those - , but it seems that they have them in their git repo too. But if the size of the objects directory is really an issue we can move the objects out of the dvcs, like the media branch, and make them available as a separate download - installers would include/download them automatically of course.

[mvBarracuda]

Wouldn't moving the objects out of the repo basically mean not putting them under version control? IMHO these files should be versioned for reasons of easy access and updating (e.g. when new graphics are added).

UH self-hosts Trac & Git, so file size doesn't seem to affect them in a way a 3rd party hosted project is affected by it.

[mvBarracuda]

I just checked out a couple of Assembla new agile features and they offer a kanban board nowadays and a bunch of other Agile tools that seem pretty neat.

I'm sold on Assembla, thanks a bunch reevaluating it Techonmage! It didn't look half that good back when I initially checked it out.

[Technomage]

Yup, Assembla looks pretty cool to me. It looks like they've done a lot of upgrading in the past few months, so I would expect some growing pains though. I did run into a couple of bugs, but their technical support staff handled those issues almost immediately so I'm not too concerned.

It looks like Assembla may be open to installing our own custom mercurial hooks on the server if we ask, so that's another nice feature. Haven't figured out the build server yet - not sure if they provide the server or if we have to provide our own external server. I'll have to do more research on that.

[Beliar]

I didn't mean moving them completely out of version control, only a different repository.

And i think Technomage didn't say that the size is an problem because of the space on the server. He said that he doesn't want cloning to take hours.