1

I have recently started a new position in a relatively small company and taken lead position on the Linux/UNIX side of things (and windows server/AD etc, but that is beside the point) in our company (before there was one guy who did absolutely everything ICT related). He is still with us and kinda my equal, but now allowed to concentrate more on the network side of things which he is probably; at least with regards experience, my superior; while I take greatest responsibility for the Linux/UNIX/Windows/(general systems and applications) side of things, (and network while/whenever he is away).

Many years ago before I joined he set up a Debian mail server (and several other internal/DMZ nodes) for our company, but due to time constraints simply did not have enough hours in the day to keep on top of updates and security for these (blame our boss; to be fair to my colleague, given the time available and knowledge he had at the time I think he did a damn good job of setting these servers up). Many years passed and eventually I took over lead for these nodes and decided to do a preliminary penetration test on the systems I now inherited responsibility over. I was able to remotely exploit bash ShellShock and show proof of concept exploits of a couple other known issues within very little time, tens of minutes in fact.

The most important of these servers is our main mail server, and most of the others in question, which are very old (version 3 I think) of Debian Linux, which there is no longer support or active repositories for (I have since set up secondaries running in parallel running on FreeBSD which are for most intents and purposes to be considered secure, but due to the rather complex setup my colleague had implemented, have not been able to completely replace).

My question is; I shall probably have to build packages such as bash and openssl and the like which have had major security holes found in them over the last few months/years, from source, as Debian have not for some time released patches for these systems and replacing them entirely is too time consuming to not have a temporary fix in place for the meantime; where would be the best place to:

  1. find which packages have had major security holes over the past few years,
  2. get source code to build them from,
  3. how to build them to avoid dependency issues (no compiler on nodes in question so shall have to cross-compile from another system, probably statically with inbuilt dependencies),
  4. how to properly test everything afterwards to make sure it works as expected and at least is no longer open to all but the most determined of intruders (I can pen test myself but realise there are several years of holes to explore).

I am mostly a Gentoo and FreeBSD user myself so haven't really kept up to date properly on Debian knowledge, but am a very experienced Linux/UNIX user in general with a good bit of programming experience, but want to be sure I don't overlook anything before potentially bricking our main mail server and several of our internal use servers.

5
  • 1
    If you're going to spend time fixing the system, you really should start at analysing the existing setup and documenting it with the view towards upgrading to a supported release, rather than adding to the technical debt by 'just' fixing up the bugs you know about. You will very likely break things by 'just' trying to replace specific packages.
    – Journeyman Geek
    Mar 8, 2015 at 11:34
  • I suspected that someone may say exactly that, seems like a rather daunting task, which will take a lot of time though ultimately is the final goal, though I need something to cover the gap whilst I work towards this (bear in mind I have 50 odd other things to keep on top of whilst thinking of a solution to this). Mar 8, 2015 at 11:44
  • 1
    Your proposed idea to rebuild every relevant package would take far, far longer than simply replacing the system. This is madness. Mar 8, 2015 at 12:15
  • You have a fair point to which I am fully aware of. To try make my position slightly more tenable I shall simply add that my colleagues setup really is quite complex and I only really hope to put in place a quick and dirty fix for the easiest and most obvious of holes in order to buy me some time to really achieve a complete replacement. As I mentioned I have already setup FreeBSD secondaries to take over as much as possible whilst I plan and implement complete replacement of function. Mar 8, 2015 at 12:31
  • I was fully aware that the reaction received was to be expected, but simply ask for my point to be not immediately soiled upon and that I have done some thinking on this issue, and would like some advice from my peers not consisting purely of dismissal. Mar 8, 2015 at 12:31

1 Answer 1

2

You have significant technical debt. You are aware that you have a problem, what you need to do now is accept that the only solution is to start again. Once you do that you will stop wasting your time on other 'solutions' and focus on getting the correct job done.

Rebuilding packages on such ancient systems is likely a fools errand which will waste significant amounts of time. You know what you have to do - do it right.

2
  • I accept your point. I had anticipated some response exactly along these lines but not that almost everyone would simply resort to this and not even try to "humour me" on my apparently not completely thought out course of action. Believe me I have for a while now been documenting and going through existing and planning and testing proper replacement, but not gotten much further than panicking over sheer complexity of task dumped on me (just to reitterate, my colleague is really quite smart and spent a hell of a lot of time coming to a fairly complex setup for me to replace with no time). Mar 8, 2015 at 12:54
  • Unfortunately there is no way to humour you. Had you say, been going from Debian 6 to Debian 7 then there may have been stop gap solutions. As it is your technical debt is so large that you're technically bankrupt.
    – user35787
    Mar 8, 2015 at 20:07

Not the answer you're looking for? Browse other questions tagged .