We take id Software's classic on the road, literally, to see how it stacks up on the Nintendo Switch.
DOOM. Nintendo Switch. Review.
An in-depth behind-the-scenes look at the game. And bass fishing.
How Ubisoft Bottled Beauty and Batshit Crazy for Far Cry 5
Join us on a chronological journey as we go through some of the highlights from three decades of Creative Assembly, the studio responsible for the brilliant Total War.
Celebrating 30 Years of Creative Assembly
Recently we had the chance to sit down with the head of Microsoft’s indie game service ID@Xbox, Chris Charla, to discuss all things Xbox, indie, and the future of the platform.
Talking Indies and Xbox One X with ID@Xbox Director Chris Charla
Your Subversion (or other) setup
TicMan
Melbourne, Victoria
5814 posts
I've been forcing our devs to go through the process of using SVN and I thought I had it implemented in quite a standard way however we're getting more and more tree conflicts and general WTF!? errors occurring with the repos that I'm wondering if I've c***ed it up initially and am now paying the price for it. We have the standard branch/tag/trunk setup.

branch - Used for the current development version of the code.
tag - Archival of various revisions or milestones in the project.
trunk - Latest production version of the code.

The process we have is that the devs check out the branch to their local machines and make code changes as required. Once it's ready to be tested on the dev server they commit their changes to branch and on the dev server we issue an svn update to get the latest copy of the branch folder. Once it's been confirmed the code changes are OK we then jump on our staging server which has a checked out copy of the trunk. We merge the changes from branch to the staging working copy and retest via the staging server that the code is working. Once it's been confirmed as working the working copy is then committed to the trunk and at a set time we then run an svn update on the production site which is a check out of the trunk.

Is this the right process to follow or is there a better way to be doing it?
11:23am 16/04/10 Permalink
system
Internet
--
11:23am 16/04/10 Permalink
Hogfather
Cairns, Queensland
5923 posts
I can't help with SVN sorry :(

As an aside, what IDE / framework / OS are your devs using?

For Windows / MS devs who hate VSS (who doesn't) but can't justify VSTS I recommend SourceGear. Its super!
11:37am 16/04/10 Permalink
Farseeker
Brisbane, Queensland
1631 posts
I like this model. Git is excellent...
11:38am 16/04/10 Permalink
TicMan
Melbourne, Victoria
5815 posts
We're using VS and chose SVN for the same reasons you said (VSS sucks and VSTS is too costly). Which reminds me, time to fire up that VS2010 download! :)

Frameworks are a mix across the sites. Our standard for new code is C# but we've got older sites that use ASP, VB.Net and even one using PHP/MySQL (we bought that company out) but they don't generate enough revenue to justify a re-write. OS for dev team is Win7 with the server platform being Win2008 + SQL2008 for the hosted sites and Linux for stuff like Jira, Nagios, SVN, SMTP gateways, etc.
11:43am 16/04/10 Permalink
Hogfather
Cairns, Queensland
5924 posts
Well it might be worth checking out SourceGear Vault then, depending on your budget.

Its a couple of hundred bucks per seat, per major version, but integrates pretty well with the Visual Studio IDE. There's a couple of quirks (there always is) but it is a pretty nice VSS alternative. The big thing for me is that the repositories are stored in SQL Server.
11:49am 16/04/10 Permalink
trillion
Brisbane, Queensland
989 posts
I was reading a great description of Mercurial (hg) and how instead of choosing to merge project branches back into your mainline you just request the changeset and if you like parts (specific lines I'd imagine) of that you can choose to accept them into your code.

11:51am 16/04/10 Permalink
Thundercracker
Brisbane, Queensland
2448 posts
Here we use another piece of software called Perforce. We currently have three branches. dev, sys and prod.

We code into dev, and when it goes across to our testers we merge the changes into sys. Then for a release to production we merge it into prod.

Sometimes if we have to do production fix we will check out the prod file, fix it there, then merge that change back to the other branches.

So at any time, the prod branch is what's in production, the sys branch is what's in test and the dev branch is a stable development environment.

Merging between branches can sometimes get a bit messy if there are any conflicting changes, but most of the time it's a non-issue.

edit: we also "label" our releases for sys, dev and prod, so we know exactly what versions of each file goes into a build. our build process first syncs to a label, then builds that.
11:51am 16/04/10 Permalink
Eds
Brisbane, Queensland
9407 posts
We used http://www.perforce.com/ where I used to work. Was quite popular over VSS

Not sure of your budget requirments but its a solid tool
11:52am 16/04/10 Permalink
thermite
Brisbane, Queensland
4853 posts
Also using SVN here, and I do get a lot of conflicts, which I just resolve by using the incoming version of the file. Not quite perfect. I do local backups of my work because I'm paranoid SVN will f*** it up anyway.
11:53am 16/04/10 Permalink
trog
AGN Admin
Brisbane, Queensland
30299 posts
TicMan, sounds mossttyyyyy right, but it seems weird to be doing a merge on staging from your dev code. You might have a reason for doing it that way.

Here's how we do it in a nutshell:

1) Devs work on code either on their local machine or directly on the dev servers

2) Once code is ready and everything has been tested as best as possible on dev, it is committed to the trunk

3) Ops, on staging, issue an svn up to update the staging server to the new revision.

4) Test on staging

5) If it's OK on staging, Ops issue an svn up on production

Notes:

- we also use branches so I left that out but the basic principle is the same (instead of svn up'ing a deployed instance of trunk, we svn switch to the branch)

- generally staging and production are always running the exact same code

- typically we would never let any merges or changes happen on the staging or production environments - if anything needs to be changed it is done in dev and the process begins anew

Here's a snippet from some of our 'corporate comms' that might also be useful:
Source Code Management is mandatory in any modern development environment. Mammoth Media utilise an approach to development that leverages the ability of SCM software to provide multiple, incomplete versions of an application known as “branches”. Each feature (corresponding to a Product Backlog entry in the ticketing system) is developed in isolation on its own branch.

When the feature has been tested and peer-reviewed, it is then merged into the primary branch of the application, known as the “trunk”. If a feature is not completed during an iteration, its branch is left unmerged and the trunk is unaffected. This allows the trunk to perform as a “known good” copy and prevents one developer from being affected by defects in another developer's uncompleted feature.

Towards the end of each iteration, the tested and peer-reviewed branches are merged into the application's trunk. At this point, a copy of the trunk is created known as the “release branch” and deployed to the staging environment. At this point, integration tests are written (in English) and executed by the development team - and depending on the project, by a QA team.
11:56am 16/04/10 Permalink
Khel
Melbourne, Victoria
14654 posts
I'm not sure if this is the right way to do it, but I've always used the trunk as the absolute latest version of the code (so the version thats in development). Then if theres a production version of the code, or a locked down version I want to be able to check out and build at any time, I'll split that off into a branch. Also use branches for prototyping stuff, or if making huge sweeping changes to the app that you want to be able to work on and check in without affecting people working on the main line (then can merge the branch changes back into the trunk when done).

And yeah, tags like you said, for milestone versions and stuff.

But then I've never really used SVN for any MASSIVE projects, so I've probably never run into problems that other people do.

Edit: Hrmm, after reading other stuff in the thread, seems I'm pretty wrong in the way I do it, but oh well.
11:59am 16/04/10 Permalink
Dazhel
Gold Coast, Queensland
1437 posts
Is this the right process to follow or is there a better way to be doing it?


Doesn't look too bad to me Tic, conflicts always increase when you're doing major surgery on one or more branches. Usually it dies down.

What Khel said though is important:
When you release to production, it's worth creating multiple release branches, e.g. 'Releases\yyyymmdd' so that you can roll out that nasty emergency fix between the time that someone commits to the trunk but releases that there's a showstopper right before deployment.
We create our tags from the release branch rather than the trunk.
12:17pm 16/04/10 Permalink
trog
AGN Admin
Brisbane, Queensland
30300 posts
Edit: Hrmm, after reading other stuff in the thread, seems I'm pretty wrong in the way I do it, but oh well.
Nope that way is what we do as well
12:46pm 16/04/10 Permalink
stinky
USA
3433 posts
I can't see any major issues with how you're doing things. I think dealing with conflicts during an integration is just something we all have to face when doing group development.

We have a fairly massive Perforce install here with 5 or 6 Tb of data across three servers. Needless to say we often have a conflict or two that need to be resolved when integrating branches. :)

Our main perforce server carries 7 million operations on a slow day, with an average of about 700 concurrent connections through the day. From what I understand from data from other large scale perforce users that I've spoken to we're pushing close to the busiest perforce site in world ( google was 2M operations/day in 2006, I've seen more recent stats, but I can't remember them off the top of my head ).

We have a 24 core 64bit linux box running 64gb RAM, 15k FC SAN for our data, and 8x64gb SSD for the revision database/journal/etc ( This is replicated to SAN disk on the fly ) and is snapshotted every 5 minutes hot, and quiesce snapshotted hourly. We also have in the works some read only replicas which can be used to take some pressure off the main server. We replicate the SAN live to two seperate locations and have hot spare servers at these locations ready to take over in minutes if needed with at worst case an hour of data lost, but more likely 30mins or less.

We often peg this machine, especially during large integrates. To track performance we monitor the load on each core, measuring how many perforce operations each core is currently processing and how much memory is spread across each. We also log each operation to a mysql database which is read and summarised by a dashboard to help track slow operations. We track file locks on the database every 30 seconds and run a timed perforce operation every minute to keep track of performance degradation trends.

12:54pm 16/04/10 Permalink
Hogfather
Cairns, Queensland
5926 posts
We have a retired 32 bit desktop that we plonked SBS 2003 on.. The machine has ~700MB of source code and doubles as the exchange server and domain controller and anything else it needs to do. Sometimes wifey does the books on it if she forgets to bring her laptop in (head-desk).

It has 2.4GB of memory or something and we do a daily backup to a removable drive which goes home with me at the end of the day if I remember to take it with me. About once a week I backup the backup to the NAS at home.

TAKE THAT STINKYPANTS
01:00pm 16/04/10 Permalink
Plasma
1030 posts
Have a read of http://svnbook.red-bean.com/en/1.5/svn.branchmerge.html if you haven't already (the whole book is great though), it may help answer a few questions too.
01:12pm 16/04/10 Permalink
Farseeker
Brisbane, Queensland
1632 posts
If svn branching and merging is difficult/painful, you should definitely try a different scm. A lot of people are finding git to be their happy place coming from svn. Git includes an svn client, so I use that at work for easy branching/merging personally.

Unfortunately even simple svn branching/merging is avoided by my workmates... everyone sees the benefit in theory, but it just doesn't happen. When they try, there is always pain.

http://git-scm.com
http://whygitisbetterthanx.com
http://nvie.com/git-model
01:29pm 16/04/10 Permalink
Eds
Brisbane, Queensland
9408 posts
Stinky are you still using compellent SANs to do your live replication? I am looking at implementing something like that (One production and another stored in a seperate data center as a failover) and was wondering how well it worked.
01:33pm 16/04/10 Permalink
stinky
USA
3434 posts
Stinky are you still using compellent SANs to do your live replication? I am looking at implementing something like that (One production and another stored in a seperate data center as a failover) and was wondering how well it worked.


Yessir .. works seemlessly, I have 3 Compellent SANs which are connected via iSCSI ( over VPN ). I'm using async replication which pushes per block changes on our LUNs to the replicas. At any given time all three of them have the same data ( give or take a second ), it also pushes through the snapshot points so even if for some reason one isn't 100% replicated due to latency/bad connection then you just roll back to the last snapshot. If you want to see it in action we can webex/meetingplace my desktop to you some time.

I've also been experimenting with ZFS snapshot and get/send to do ZFS replication which works really well but needs some good snapshot management scripts behind it to ensure they don't get out of hand.
01:44pm 16/04/10 Permalink
Pinky
Melbourne, Victoria
5463 posts
+1 to Git but it takes some getting used to if you're from a CVS/SVN background

This article really helped me understand it.

http://www.joelonsoftware.com/items/2010/03/17.html
01:50pm 16/04/10 Permalink
Dazhel
Gold Coast, Queensland
1438 posts
This article really helped me understand it.
http://www.joelonsoftware.com/items/2010/03/17.html


Nice article. I hadn't taken the time to understand it before coming from a CVS/SVN background. It just looks like instead of a central repository and multiple working directories, everyone's working directory becomes a repository in itself.

I've heard this is kinda the way that the Windows source control process works internally (though they don't use Hg or Git obviously). Because they've got so many developers and such a large code base the changes have to pass quality gates and changes propagate from the individiual component teams right up to the top of the hierarchy and then down again.
02:43pm 16/04/10 Permalink
TicMan
Melbourne, Victoria
5818 posts
Thanks everyone, absolutely invaluable information about how everybody implements their own different version control software and then how the process of using it is. stinky's example is just crazy but a good testament to software like Perforce which I'll have a poke at. Git has been mentioned numerous times so I'm wondering if that could perhaps be the next bit of software I check out.

It's also heartening to know that conflicts do occur and are seemingly just a way of life. For me and the developers we thought it was a huge cluster f*** when a conflict popped up but we stumbled through it somehow and got the code pushed out in the end. I have a suspicion some developers circumvented SVN and just started to make changes directly without following the process which has led to the situation I'm in now with the trunk not being able to be committed to as there is huge conflicts between them.
02:47pm 16/04/10 Permalink
trog
AGN Admin
Brisbane, Queensland
30309 posts
I use the same setup as stinky for my CLOCK
02:49pm 16/04/10 Permalink
Eds
Brisbane, Queensland
9409 posts
If you want to see it in action we can webex/meetingplace my desktop to you some time.


Yes , Yes I do. That said I will be meeting with compellent in the few weeks as well. Basically what I am doing is setting up a bunch of blades running vmware vsphere and our data. I want to replicate the data of our EDRMS system to a recovery system for DR and BCP.

Thats if I can set it all up for 350k
06:40pm 16/04/10 Permalink
Dazhel
Gold Coast, Queensland
1440 posts
I'm using async replication which pushes per block changes on our LUNs to the replicas.


Hey stinky, how big is the pipe that your replicate across and how much data (avg latency, etc)?

We're attempting to set up a DR replication solution using DoubleTake from Darwin to the Goldie for about 10 servers. At the moment we've got a 2Mbit link but probably looking to up that in the future. For data replication we looked at a Dell Equalogic PS4000 series iSCSI array as part of a VM consolidation exercise recently but the bean counters baulked at the price of setting them up at each site so it's back to local storage. =sadface=
07:33pm 16/04/10 Permalink
stinky
USA
3435 posts
Yes , Yes I do. That said I will be meeting with compellent in the few weeks as well. Basically what I am doing is setting up a bunch of blades running vmware vsphere and our data. I want to replicate the data of our EDRMS system to a recovery system for DR and BCP.

Thats if I can set it all up for 350k


We have a pretty decent vsphere install now, 5 servers running 34 virtual servers all served from the Compellent. The Compellent plays really well together with vmware. I reckon you'll be pushing it to get that for 350k, especially if you want to replicate.

Although all you need at the remote end of the relication is a single controller hooked up to a shelf of 2tb disks ( probably not even full ), or you could try get them to offload some smaller disks cheap if they have excess stock.

Are you dealing with Regal IT for the compellent? or have they got a local brisbane reseller yet? Feel free to tap me if you need a pair of eyes on configs etc.
11:41pm 16/04/10 Permalink
stinky
USA
3436 posts
Hey stinky, how big is the pipe that your replicate across and how much data (avg latency, etc)?

We're attempting to set up a DR replication solution using DoubleTake from Darwin to the Goldie for about 10 servers. At the moment we've got a 2Mbit link but probably looking to up that in the future. For data replication we looked at a Dell Equalogic PS4000 series iSCSI array as part of a VM consolidation exercise recently but the bean counters baulked at the price of setting them up at each site so it's back to local storage. =sadface=


we've got dual gig WAN links to the offsite locations. However the bulk of the data is replicated and it's just getting changes, I reckon a 2Mbit link would be okay. the dual gig is because we have a lot of other traffic going on as well, I set it to run at a max of 200mbit during office hours.
11:44pm 16/04/10 Permalink
Pinky
Melbourne, Victoria
5470 posts
Dazhel, just in case you didn't click through to this link, read this too: http://hginit.com/

Or TicMan or whoever
12:02am 17/04/10 Permalink
Dazhel
Gold Coast, Queensland
1446 posts
Yeah I was halfway through a 'why the hell is git the best thing since garbage collected memory?' and I clicked through to that and went through the awesome tutorial. No need to post that now. :D
12:27am 17/04/10 Permalink
Infidel
Netherlands
3206 posts
I did not read all the replies but we also edit trunk and then svn commit the trunk also cfengine is nice
12:27am 17/04/10 Permalink
simul
Brisbane, Queensland
806 posts
+1 git or even mercurial

Once you go git you never go back!
12:44am 17/04/10 Permalink
hast
UK
1171 posts
you should be doing all your merges/etc on developer machines and creating builds by checking out the latest on trunk or a branch. don't be checking out on a staging machine and then doing merges from there. that is just crazy.

our setup at work for only 4 devs and most of them are part time on the project :

we do all of our development on trunk and try and do regular checkins. break down your tasks into smaller chunks that can be done in less than a day and safely committed to trunk without breaking the build.
when we do a major release we merge from trunk to a release a branch.
we have a ci box that runs builds by checking out the latest on a release branch or trunk. it also automatically deploys to our staging release server or our staging trunk server. (hudson is very freakin cool: http://hudson-ci.org/)
we have a script that takes a release generated from our ci box and automatically deploys them on our live cluster. (all builds from hudson have a number so we just go: ./release.sh <number> on a live server and bam we have the new version)
we do bug fixes by doing the fix on the release branch and on trunk so merging back from the release branch into trunk isn't necessary.

this leaves only merging by people working on trunk.

last edited by hast at 05:38:48 17/Apr/10
05:29am 17/04/10 Permalink
Insom
Brisbane, Queensland
3346 posts
don't much see the point of checking something in until it's done
11:42am 17/04/10 Permalink
mooby
Brisbane, Queensland
5414 posts
we use mecurial to complicate things more. i dont like it, but put up with it. moving to tfs soon. in an agile + xp env, first check in wins. others should merge.

comes down to training imo. at first i was getting grief for not checking in properly. but if you dont know / dont have a procedure, its managment issue.
11:55am 17/04/10 Permalink
mooby
Brisbane, Queensland
5415 posts
don't much see the point of checking something in until it's done

theres a philosophy, dont check in till it builds.
11:56am 17/04/10 Permalink
Hogfather
Cairns, Queensland
5937 posts
I don't think I ever want to be "big" dev. If we do I'll certainly not want to be involed in development at that point.

Having at most 2-3 devs on a project at a time allows for nice things like exclusive checkouts. I'm old I know, its very cool that it works, and I'm basing my opinion on very old systems where it didn't work well, but merging code still just gives me the heebie jeebies.
01:00pm 17/04/10 Permalink
Insom
Brisbane, Queensland
3348 posts
yeah i'm glad our dev support people have their heads around the VSTS business because i'd make a meal of it

even now there's a fair bit of rootin' around involved in hotfixes
03:02pm 17/04/10 Permalink
Dazhel
Gold Coast, Queensland
1448 posts
Having at most 2-3 devs on a project at a time allows for nice things like exclusive checkouts.


Exclusive checkouts are nasty though, but I guess for some file types like binary document formats they're a necessary evil.

Why the hate about merging code though? If I'm working on a change and need files A & B and someone else is working on a change and needs files B &C whoever has B checked out will block the other developer.
This is why Visual SouceSuck was so bad as it encouraged whoever got in first to keep an exclusive check out on the vs project file until his changes were done. That and it was riddled with repository integrity related bugs so much that it was a gamble each day whether your source code tree would be versioned or destroyed.
03:21pm 17/04/10 Permalink
Dazhel
Gold Coast, Queensland
1449 posts
don't much see the point of checking something in until it's done


Define "done" though.

There's often times when I'm working on something that's 50% completed and builds fine, but I know it'd cause people grief if I checked it in because they might end up chasing bugs I caused if they see it too early.

If I'm at a crossroads and want to investigate something to see if it'd pan out, and if not I'd revert to that 50% done point, it'd be nice to be able to check in the code to have a point in time to revert to.

From what I can tell, Mercurial and Git support this easily with their private repository model, however subversion doesn't unless you go to the hassle of creating a branch and the manually migrating all the halfway completed code onto that branch, then merging back when finished.
03:29pm 17/04/10 Permalink
Hogfather
Cairns, Queensland
5940 posts
Dazhel - SourceGear Vault has a functon called "shelve" :
Shelve: Shelve lets you store changes in the server without checking them in. These "shelveset" can be shared among Vault users, or restored for later checking.

I use this pretty much once a week.
06:28pm 17/04/10 Permalink
Dazhel
Gold Coast, Queensland
1450 posts
TFS has shelvesets as well.
Hog: What made you buy Vault?

I'm genuinely interested as to how the product survives when there are so many other products out there that filling that niche (a lot of them are free per user) and have the checkmarks like visual studio integration etc etc. (TFS, Perforce, Subversion, Mercurial, Git...)
06:53pm 17/04/10 Permalink
Hogfather
Cairns, Queensland
5941 posts
I went with Vault when it was just me doing my thing and I wanted a VSS alternative - its free for a single developer. I tried Subversion and it just didn't work for me - coming from VSS I found the VS IDE integration (at least at the time, this is 2007 or something) to be woeful.

The licensing is only a couple hundred bucks per seat, per major version. We don't have enough developers for it to be woreth changing - I'm happy enough with it that I haven't been motivated to look into options to be honest.

If I spend a day f*****g around with alternatives rather than developing production code etc then its cost us the next major Vault release :)
07:38pm 17/04/10 Permalink
Insom
Brisbane, Queensland
3349 posts
Define "done" though.

well i guess that depends on your specific processes - but it was in response to someone breaking up a task into smaller bits that could be checked in at the end of each day - why?

if it's for backup purposes then use shelvesets or equivalent

then again we don't use the concept of each developer having his/her own repository, maybe it makes more sense for that
07:44pm 17/04/10 Permalink
hast
UK
1172 posts
Insom: there is definitely trade offs and it is not always the best way for every situation. A lot of people who practice agile already try and limit the complexity of tasks. The advantage of having smaller units is you often have less merging problems and also less time management problems because others can easily observe how much is done rather than only having one set of eyes claiming x% is done.
09:29pm 17/04/10 Permalink
hast
UK
1173 posts
I know one software house in Brisbane had a check in gong or something similar and when it sounded you were meant to have either checked in or reverted your changes. I think that might be a little extreme but it does force developers to break up their tasks and plan better.
09:33pm 17/04/10 Permalink
Dazhel
Gold Coast, Queensland
1451 posts
I know one software house in Brisbane had a check in gong or something similar and when it sounded you were meant to have either checked in or reverted your changes.


That sounds ridiculous. Who is going to lose a bunch of work or break the build for your team members just because a Michael Scott style manager likes to play games with musical instruments?
11:08pm 17/04/10 Permalink
stinky
USA
3438 posts
Dazhel - SourceGear Vault has a functon called "shelve" :


Perforce introduced this in 2009.2, I don't know if any of our devs are using it yet, but it offers some cool options for peer review etc.
02:56am 18/04/10 Permalink
Thundercracker
Brisbane, Queensland
2449 posts
For my own personal projects I use SVN, mainly because it's free and it's explorer/visual studio integration is quite good.

I use AnkhSVN for VS and tortoise svn for explorer integration. In fact I find these tools superior to anything i can currently find for perforce.

Yeah perforce has some new shelve like addition, but it's currently poorly supported by the graphical interface that we tend to use.
02:28pm 18/04/10 Permalink
Dazhel
Gold Coast, Queensland
1455 posts
I've used VisualSVN for a few years now mainly because it was/is pretty cheap and AnkhSVN had some stability issues early on. I've heard Ankh is much better now.
03:27pm 18/04/10 Permalink
Fizzer
Brisbane, Queensland
722 posts
Ticman

There should be no issue with branching, working out of the branch and then committing once you're done.

Are you merging from the command line? One of the common pitfalls is if you don't take into account the revision trunk was at when you branched. You want to merge your branch into trunk but only starting at revision that it was branched.

In the branch:
svn log --verbose --stop-on-copy (will give you like rXXXX) which is the revision # it was copied at

In the trunk:
svn merge -r XXXX:HEAD svnurltobranch
11:05am 19/04/10 Permalink
stinky
USA
3439 posts
Yeah perforce has some new shelve like addition, but it's currently poorly supported by the graphical interface that we tend to use.


P4V (2009.2) ... right click 'shelve files'.
11:57pm 19/04/10 Permalink
system
Internet
--
11:57pm 19/04/10 Permalink
AusGamers Forums
Show: per page
1
This thread is archived and cannot be replied to.