Announcement

Collapse
No announcement yet.

Security? I'm sorry, I have no clue what you're talking about (Long)

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Security? I'm sorry, I have no clue what you're talking about (Long)

    Ok, hi guys, first story posting here. This one isn't technically a customer, more of a contractor, but definitely tech related...
    I work for (Almost-Ivy-League-College (AILC from here on)). More specifically, I work for one department, as the "tech guy". This means I do everything from write the software to connect our website to our e-commerce provider to changing the toner in my Overboss's printer (Overboss = Boss's Boss). Needless to say, I'm also the only one in the building who has any concept of technology.

    Anyways, we're paying a company to do upgrades to our website, mostly because there's only one me, and I'm currently working on afore-mentioned e-commerce stuff.
    We give him access to the server, and in an attempt to keep it as secure as possible (I also double as sysadmin) I ask them to send us an ssh key, so that I don't have to give him a password, and can track their computers if necessary.
    In response, I get sent 6 files. 3 public keys, and 3 private keys. For those of you who don't know, private keys are named such because they're never meant to leave the computer that they're created on. They're private.

    We get that sorted out, and after some problems with my boss not knowing what a ` (backtick/grave) character is, we get the contractor access to the server. Mind you, this is the production server. We have testing servers, and I'm sure they do as well. We gave them access so that they could pull what data they needed without having to take a 2GB file over a slow network. Most of the production code is small, but we're hosting a lot of photos (comes with the sector).

    I come in the next morning to notice that one set of our logs had been deleted. Not maliciously, but probably in "clean-up". Ok, first of all, I'm a web developer by trade as well, and I'd never mess with a production site. Never ever ever. All things are done on a test site, and then when everything checks out, pushed at once up to the production site.

    Anyways, I immediately go into search-and-destroy mode. And, lo and behold, the developer has uploaded several things directly into the webroot of the server. A script designed to create an archive of the necessary files, a script to unzip another archive containing who knows what, and a copy of phpmyadmin. We gave the developer ssh access for a reason. When creating an archive, you create it outside the webroot, over ssh, probably using tar -czvf. When unzipping an archive, you do the same. And had the developer asked, we already had a copy of phpmyadmin running securely on another port.

    Worse yet, I find a database dump sitting in the webroot. A complete dump. Containing all of our users password hashes. Among other things of course.

    I shut the developer's access down faster than you can blink. That's too far. I cleaned up after him, and put in a call to the company, asking for a conference with him. That was yesterday. Haven't heard from them yet. I'll keep you updated.

    PS. Sorry for the long post, but I had to rant. I understand not being paranoid about security, I'm the one being paid for that, but to not even think about it? I'm just glad this developer isn't touching our e-commerce code. I don't even want to think about the potential security issues there.

  • #2
    unfortunately a lot of people just do things the way they were taught to do things, they don't think about why they do such things, and then when they actually meet someone who actually knows his stuff... well... the above, or worse happens.

    I just hope they didn't "clean up" anythign irreplaceable and it would be so amusing to listen into the conversation you'll have with them

    oh, and before I forget, welcome to CS!
    "You can only try so hard to look like you are working before actually doing your work seems easy in comparison" -My Boss

    CW: So what exactly do you do in retentions?
    Me: ummm, I ....retent stuff?

    Comment


    • #3
      And a lot of people just don't think about what implications their actions can take.

      ...and a lot of developers don't know the first thing about the environments their code runs in, let alone how to interact with it safely.

      ...and some people are just too plain lazy to care. Those are the ones you need to take the cattle prod to and take access away from.
      "If at first you don't succeed, skydiving is not for you."

      Comment


      • #4
        It's not paranoia if you're right....

        The fact that a) he is suddenly hard to reach and b) he deleted a logfile (or more), plus the fact that he was obviously in a hurry (failed to delete his db dump) suggests to me that he knew damn well what he was doing.

        I would suggest that you CYOA at the very least and bring this to the attention of the boss, the overboss, any legal beagles you guys have, and anyone above you who is responsible for IT -- even tho the immediate supervisors may not have a clue what you're talking about -- it's best that they hear as much as possible as soon as possible. If need be, simplify it for them:

        - This guy had access to information he shouldn't have. His access has now been cut off completely.

        - This makes him a security risk, even with the info he has now.

        - This could cost the company money. Lots of it. (main point for the bosses)

        - This guy's actions are ultimately the responsibility of his own employer.
        "For a musician, the SNES sound engine is like using Crayola Crayons. Nobuo Uematsu used Crayola Crayons to paint the Sistine Chapel." - Jeremy Jahns (re: "Dancing Mad")
        "The difference between an amateur and a master is that the master has failed way more times." - JoCat
        "Thinking is difficult, therefore let the herd pronounce judgment!" ~ Carl Jung
        "There's burning bridges, and then there's the lake just to fill it with gasoline." - Wiccy, reddit
        "Retail is a cruel master, and could very well be the most educational time of many people's lives, in its own twisted way." - me
        "Love keeps her in the air when she oughta fall down...tell you she's hurtin' 'fore she keens...makes her a home." - Capt. Malcolm Reynolds, "Serenity" (2005)
        Acts of Gord – Read it, Learn it, Love it!
        "Our psychic powers only work if the customer has a mind to read." - me

        Comment


        • #5
          Quoth Shishire View Post
          In response, I get sent 6 files. 3 public keys, and 3 private keys.
          WTF? That, right there, would have had me putting the brakes on. I can see sending three different public keys, since different servers use different formats (which is frustrating in itself). But to include the private key files? Idiocy of the highest order there.

          Quoth Shishire View Post
          We have testing servers, and I'm sure they do as well.
          You are most generous in that assumption. I've seen, and dealt with the developmestuction environment myself. It's not pretty, and it happens far too often.

          Quoth Shishire View Post
          Worse yet, I find a database dump sitting in the webroot. A complete dump. Containing all of our users password hashes. Among other things of course.
          If you have good password policies in place, and those hashes are actual hashes (md5sum, sha1, etc), then the other guy's ability to do anything with those password hashes is pretty nearly zero.

          I was going to try to find a way to defend some of what he did, but can't. If he had ssh access, then he had scp access. Putting the db dump in the webroot to download it might be acceptable if you had placed it there for him to download via wget. For him to do it, though? Someone needs a lesson in the use of their tools.

          I'm trying, but can't defend him. Hell, I don't even do it here. Our offsite backups are downloaded via rsync/ssh, and the local copies of them are *not* stored anywhere near the webroot.

          Quoth TheRabbi View Post
          ...and a lot of developers don't know the first thing about the environments their code runs in
          Careful, there. That's a pretty wide brush, and would get a lot of people up in arms were you to change a few bits and say something like "and a lot of retail workers don't know the first thing good customer service."

          I got into a similar discussion many months ago here. Suffice to say it was not pretty then, and wouldn't be pretty now.

          Comment


          • #6
            Quoth Pedersen View Post
            Careful, there. That's a pretty wide brush, and would get a lot of people up in arms were you to change a few bits and say something like "and a lot of retail workers don't know the first thing good customer service."
            Okay, you've got a good point. s/a lot of/a larger number than one might think of/;

            Being a developer and a sysadmin, and having come by the latter by necessity brought on by being the former*, it always amazes me to find (say) a web developer who can't install a webserver or doesn't know how databases are supposed work. And I've found them much more often than I'd ever have expected to.

            (I've been lucky in my career, though, to never be working with those people.)

            * Though I will say that I kinda like being a sysadmin more than being a developer now...
            "If at first you don't succeed, skydiving is not for you."

            Comment


            • #7
              Can I ask something? Since I'm more of a hardware person and not a dev...

              Why would the dev guy be given authority to the Production server at all, until after he's tested/debugged his stuff on the Testing server?

              I thought the idea of Testing is to push new code, run it for a while, let it do all the things it's new to, find bugs then push beta or charlie dev to the production?
              In my heart, in my soul, I'm a woman for rock & roll.
              She's as fast as slugs on barbituates.

              Comment


              • #8
                Actually, there's a number of possible scenarios that would account for it. The simplest one would be to allow him to copy the configuration of the production server into his own test/qa environment. It's easy to be told what you need to do to mimic production, but that rarely is complete enough to actually mimic production.

                Another scenario: The dev was hired to upgrade the existing code, not to write all new code. Depending on the OP's environment, the only copy of the code might exist on the production server. Ideally, it should be in version control and in backups, but many shops run without at least one of those (and far more shops than you would care to think run without both of those).

                Another scenario: While doing dev work, sometimes (though this is exceedingly rare) you have to see how the code runs in production to understand what it's doing. It could be using some obscure bit of syntax (if the original code is written in Perl, then "obscure syntax" is almost the rule of the day), or doing something in a way that doesn't make sense until you can see how it is interacting with other systems.

                So, gettimg the dev access to production is not at all unusual.

                Comment


                • #9
                  Well, I know it is way off topic, but I want you to stop and think about something. How many times do you get asked for your Social Security Number, and just say it out .. while in a public place? I've seen/heard it happen in Banks, stores, all kinds of places. People won't give a telephone number, but a Social Security Number? Sure!

                  I went to the DMV, the person there says. "Now I will need your SSN number, but do NOT say it out loud. I need your actual card, or something with your SSN number written on it."

                  Security can be a joke, and on the internet people seem to have a very lax way of looking at things. Your personal security is important, treat it as such. I'll get off my soapbox now.
                  Engaged to the amazing Marmalady. She is my Silver Dragon, shining as bright as the sun. I her Black Dragon (though good honestly), dark as night..fierce and strong.

                  Comment


                  • #10
                    Quoth Mytical View Post
                    Well, I know it is way off topic, but I want you to stop and think about something. How many times do you get asked for your Social Security Number, and just say it out .. while in a public place?
                    Bear in mind, Developers, sysadmins, db admins, netadmins, and others are /supposed/ to be security conscious. If I were to do such a thing, I'd be fired instantly. In fact, I'd have been fired instantly for the private key transgression. Granted, my company's current change of heart security wise was because we got pwnt by some turkish script kiddy, but still.
                    Coworker: Distro of choice?
                    Me: Gentoo.
                    Coworker: Ahh. A Masochist. I thought so.

                    Comment


                    • #11
                      Quoth Mytical View Post
                      Well, I know it is way off topic, but I want you to stop and think about something. How many times do you get asked for your Social Security Number, and just say it out .. while in a public place? I've seen/heard it happen in Banks, stores, all kinds of places. People won't give a telephone number, but a Social Security Number? Sure!
                      I'm always *always* shocked by how lax people are about this. I'm not a paranoid security wonk by any means, but I don't say my SSN out loud in public, I don't write it on anything that isn't going directly to someone who needs it for a good reason- and that I trust. I don't have a file with passwords and account numbers on my desktop, and I never even *considered* putting it on my lappy. Those are way too easy to steal.

                      I keep an actual ledger(double entry accounting system gooooooo) which is where any actual account numbers are. The account numbers are not in a form that is easy to decode unless you know exactly what you're looking for, and the ledger is not kept in an easily accessible place. My passwords are all in my head, though I have backups... in a safe deposit box... at a bank.

                      ...maybe I am a paranoid security wonk.
                      Character flaws aren't a philosophy -Scott Adams

                      Comment

                      Working...
                      X