This is one reason us corporate software pushers can't use github and the likes. We need stuff to be hosted on a network we control. Each hour we lose is potentially multiplied by the number of staff as we would push and pull hundreds of times a day. Someones head will roll if were down for more than a couple of minutes at a time.
It becomes just as much of a PITA importing projects github and maven for example as well.
We actually ended up sticking with svn and trac and doing manual mirrors of every source dependency we pull in to protect our asses. SVN is kept only due to externals support (even though all our externals are internal!). Our head rev is over a quarter of million revs, 37gb of data and the repository has been online since 2006 to give you an idea.
I'm aware of github enterprise btw - its actually easier to build our own using trac as we can support it end to end, its a crap load more configurable and we can scale it up and run failover nodes easily using svnsync and pgsql replication. We have our own plugins and reports plugged in using reportlab as well. And its not atlassian's crap either.
Using github as an excuse to stay with svn seems rather ill informed.
You wouldn't lose hours if github is down. You might lose your ability to deploy (if you are deploying from github) but you could setup any other remote you like, and deploy from that.
I'm not sure what the 'size' of your repo has to do with anything...
I think the killer feature of git -- which applies to any other DVCS -- is that you're not dependent on github if you're well prepared. There's absolutely nothing stopping you adding redundant remotes so you always have somewhere to push/pull changes and deploy from.
svn is fine for very small, highly co-ordinated teams which is rare in a corporate environment. It's decidedly not fine for complex merging, development outside of the private network, or maintaining multiple versions of the same codebase (customer installs) among other things.
i used to use them extensively. unless something has radically changed, they cost nothing in the svn repository but when you pull them down, you have multiple copies on disk which takes double the space because of the way a branch appears like a separate folder in your working copy. am i missing something?
It is interesting, I have heard several times the comment about svn branches being expensive. But, that is not true, they are checking out the whole branches directory!
Are you checking out all the branches? I mean the actual branches subdirectory? If you do that, it is going to be expensive. But you shouldn't do that. Just checkout the trunk and switch to branches as needed.
You have to maintain a full copy of the file tree locally per branch, it's a pain that gets worse as your project size grows. With git, switching between branches is near instantaneous.
sorry but that is a very long time. Branching is something I do dozens, if not hundreds, of times a day (creating, checking out, deleting, etc).
svn doesn't fit the sane modern workflow and everything it does can be done by git in a less painful manner. I think the only use case I can think of for svn is a very binary heavy repo that needs global locking.
We'd lose our deployment, integration tests, continuous integration, ability to pull other team changes from the master and code review platform. That's a big chunk of our dev infrastructure.
Size is mentioned as to be honest we moved a load of stuff to git in 2011 as a test case and it couldn't handle it. We have some test fixture data files (big ones) that it choked on. Not only that, you can't compose software easily without introducing external tooling.
> We'd lose our deployment, integration tests, continuous integration, ability to pull other team changes from the master and code review platform. That's a big chunk of our deb infrastructure.
You'd lose all that from using git? Why? Are you saying if you switched to git without switching any of your svn infastructure? We aren't talking about github downtime anymore are we...
> Size is mentioned as to be honest we moved a load of stuff to git in 2011as a test case and it couldn't handle it.
A repo I interact with daily is now pushing 54gb, it contains large test fixtures that have HD videos, and images in it. I've never had any issues. (It takes a while to copy it around...)
git is decentralized. Any sane sysadmin wouldn't use github as a 'central' point, but set up an own remote to which you push. Then every once in a while from that remote, commits are pushed to github.
Alternatively you can set up an alias (i.e. "all") for multiple remotes and just push to them in one hit. Useful when I'm pushing code to Heroku and GitHub simultaneously.
Well it doesn't. We package every rev that's made as part of our integration process. Then we choose one and deploy it (using our auto deployment software which farts out packages across 100-odd machines, per instance..).
> This is one reason us corporate software pushers can't use github and the likes. We need stuff to be hosted on a network we control.
cringe This is very corporate reasoning in the most pejorative sense of the word.
> Each hour we lose is potentially multiplied by the number of staff as we would push and pull hundreds of times a day.
You clearly haven't used git. Even if github were to disappear, it would be absolutely trivial for a team to switch to an alternative central git repo for everyone to work through.
> Even if github were to disappear, it would be absolutely trivial for a team to switch to an alternative central git repo for everyone to work through.
Except for all the tooling built into github that would have to be replicated locally somehow.
> Except for all the tooling built into github that would have to be replicated locally somehow.
What tooling? The issue tracker and code comments? I imagine the uptime requirements aren't >99% for those, but if you absolutely need them all the time, that's what github's enterprise service is for.
The issue tracker uptime is higher than the VCS in a corporate environment. Its more an ALM tool for us as it has use cases, defects, project planning running off it. That means the front line support, dev team, project management team, business analysts and management all use it 24/7 (in our case from 5 different countries).
github enterprise is a crappy solution for the above as well and is hard to support and expensive too.
Excuse my ignorance, since I don't work in the corporate world. But isn't one of the central points of a distributed version control system like git that you don't have to host it at all? (Of course, you still need a network that you're in control of.)
Sort of. You have "remotes" which you copy your own repo to. So in theory you could just have everyone push code to everyone elses workstations.
It is probably prudent to have some central location to push code to though, especially if you want people to work on it outside of your company LAN.
Best solution is probably to have a server you control and have a copy on somewhere like github and keep them in sync. That way you don't have to worry about DDOS of github and you don't have to worry about what happens if a drive dies on your server etc.
Yes. I do. Unfortunately it doesn't support your average corporate workflow and requirements of centralisation very well (it can do it but its not its primary use case). The desktop tooling is also crappy and hard to get non-code-monkeys using it. We use svn for documentation as well and our windows users quite happily edit word documents collaboratively on top of it.
I stand baffled. Always thought Word and Excel files were binary files, thus almost un-mergeable in any useful - collaborative - manner.
Sorry for my misunderstanding.
I'm sorry with the poor assumption... But you started this thread with "Github downtime is the reason we can't use git internally" <--- This illustrates either you don't understand git very well. OR you don't understand logic very well.... I was giving you the benefit of the doubt on the latter... :\
Which I must say is also insanely priced, particularly since you have to pay for everyone with a user account.
Which means if you're wanting to use GitHub issues to replace some bugtracker the non-developers need access to you're paying full cost for everyone who might need to update a bug once or twice a year.
I understand that they're trying to get everyone to use their online offering, but a lot of organizations wouldn't consider it due to a combination of not wanting to host their code externally, and GitHub's ongoing uptime issues. They might go for GitHub Enterprise, but it's crazily priced compared to other similar software (e.g. Atlassian Stash+Jira).
Not really a good justification for not using GitHub. If your SVN server goes down, you're in a lot more trouble than the few times GitHub is unavailable. At least each user has a full working repo and you can always prepare alternate remotes to share commits. You are apparently already aware of GitHub for Enterprise. I suppose your organization is in full control of your data centers, private cloud, etc, but rationalizing about svn and trac completely stumped me as to your point about "corporate software pushers..."
With SVN each user has a local working copy. You can't commit if the central server is down, but they're not exactly dead in the water either. You can keep working locally. If your SVN server is down for DAYS it might get to be a problem.
Does Github's web-presents actually take down their back-end/client access? I don't know, but I would assume they would use a different server/servers/DNS address for that stuff?
I think that's the exception rather than the norm. Since we're sharing anecdotes, every corporate shop I've worked in tends to stand up services, then once they are working they are not monitored or thought about until there's a problem.
Well both of our experiences are anecdotal aren't they. And having a server running doesn't mean that everyone has access to it, one of the issues I've faced is getting approval for VPN access (took a month IIRC) which prevented access to the SVN repo, would not have been an issue with Github.
And that org's infrastructure team could probably claim 5 nines uptime as well.
(Good) corporate services wouldn't have had slow VPN provisioning.
This has nothing to do with Github, other than the fact that Github would let you end-run all rules about data protection, backups, centralized administration of user access, and everything else that you don't individually have to worry about, but the management and IT at the company you work for does.
This is exactly the sort of thing I'm talking about. Let the team working on the code decide who has access to the code. Not some person in a different part of the country who has no idea what the code does.
Actually it would be even worse as we can't risk people pushing our code to their private repo so Mr Firewall goes "no way". Another problem: we dont know which repo we're pushing to at a network level.
Code theft is a major problem. Not because the code is crap and we're embarrassed (unlike VMware and Microsoft code leaks), but because its valuable.
I really wish they'd release geographic information on who's launching these attacks. On the bright side, a Saturday @ 7 eastern isn't near as annoying as something during the workweek.
It's really quite a shame at how little can be done to find the perpetrator. I suppose contacting law enforcement wouldn't do much either, unless it's a significant persistent threat that would warrant a large scale investigation.
Based on the FBI taking 50 doors related to the operation payback ddos and yet not charging anyone it appears the US Attorney's office isn't sure its even a crime.
Possibilities mentioned so far:
(1) Fame for attacker
(2) Excuse for other problems at GitHub
(3) Competitors
Not sure I find any of these is more convincing than the next or very convincing at all.
GitHub seems to be one of the least offensive businesses around but still gets attacked on a regularly. It's popularity might make it a target but don't those running botnets have better targets? And wouldn't they publicize their exploits?
And I can't believe GitHub would use it as an excuse. A DDoS attack has clear evidence that their entire devops team and many of their suppliers would have first hand knowledge of. If word leaked out it would be pretty embarrassing.
Lastly would a competitor risk destroying their entire business if found out. People are irrational and stupid but that would be crazy.
Anyone counting how many DDoS attacks github saw this year? I don't doubt they are having issues, but at this point it seems like they are blaming every outage on a DDoS
Extortion is one of the better possible reasons for their DDoS. Eventually, when the money doesn't show, its in their best interest to stop attacking. While new extortionists will keep showing up, they won't likely come back.
Ideological or government attackers, on the other hand, can't be reasoned with and can only be expected to escalate in the future. If that's who Github is dealing with, then they'd better figure out how to actually deal with DDoSes sometime soon, because they will only increase in size and frequency.
In light of this it would be nice if github offered a feature to push repo's to s3 or something every time they receive a push, so you could use that repo until they get back up.
You know what I want? Transparent failover somehow baked into git remotes.
you <--> (X <--> Y)
If you push to and pull from X, and X should be unavailable, automatically start using Y. When X is pushed to, check with Y first to keep in sync. There are ways to push to multiple remotes by defining multiple remote URLs, but then it's a harder problem of dealing with downtime.
It's really easy to push to two remotes. I have a tfs service account setup and using git and also a github account. I like tfs's project and issue tracking better but want the public repo available on github.
git add remote <name> <url>
git add remote <name> <url>
then create a shell script that checks both remotes for the latest if one is greater than the other merge the two if they are the same then push to both. Then just always push with that script.
Just how I've handled it so far. That said a autoMagical way of handling it built into github/git itself would be cool.
As I said, it's doable with scripts and whatnot, but transparent and built into git would be awesome. I see it as something you just set up on a remote repository server. No need to make N developers aware of it, or aware of the script, or resolving the script breaking, or having a ton of implementations of said script across different teams and companies.
Doesn't help if you use GitHub for issues, of course, but hey!
It's back. I was able to watch the graph change and see it reflected in my own experience of trying to load a GitHub page. Great job on the Status site, GitHub!
I first noticed the top graph change, and a minute or two after it went back up towards 100%, they updated their status message to "Minor service outage". Now it's been moved to the history: https://status.github.com/messages
Oh that explains. Seems it started half hour before they posted the update.
I was trying to make pulls just a minute ago and it was incredibly slow, but the status.github was just showing "we're investigating the high rate of drop packages".
The only real thing is to encourage behaviors that stop people from having their machines compromised. Most of the DDOS attacks are masses of compromised machines.
The attackers would just find such a user and spoof their IP. DDoS is a hard problem to solve, and it's a shame that so many ISPs and datacenters don't work harder to prevent spoofed traffic. On top of this, they'd still need routers and switches in front of their machines big enough to handle the traffic from the attack plus the load of trying to filter out the good traffic (this kind of hardware is quite expensive).
Wow, Bitbucket must be getting desperate... Haha. This is annoying, even though it's a weekend, I've got work to do and the fact I can't pull down some changes I made on a machine at work to my home machine is highly annoying.
It becomes just as much of a PITA importing projects github and maven for example as well.
We actually ended up sticking with svn and trac and doing manual mirrors of every source dependency we pull in to protect our asses. SVN is kept only due to externals support (even though all our externals are internal!). Our head rev is over a quarter of million revs, 37gb of data and the repository has been online since 2006 to give you an idea.
I'm aware of github enterprise btw - its actually easier to build our own using trac as we can support it end to end, its a crap load more configurable and we can scale it up and run failover nodes easily using svnsync and pgsql replication. We have our own plugins and reports plugged in using reportlab as well. And its not atlassian's crap either.