Skip to main content

View Diary: Data Storage and the NSA (121 comments)

Comment Preferences

  •  The project I work on can have a petabyte (3+ / 0-)
    Recommended by:
    JDWolverton, JML9999, Cliss

    in a single storage cabinet, and that's with just normal compression, not with something like Dedupe, which may increase compression by major factors depending upon the type of data.

    Prices drop, and for large customers who can buy multiples of these, they do get price breaks.

    BTW, the server nodes can have a terabyte of RAM in them, which is four times what the diary lists, and we've got other servers which can easily hold more.  Have multiple processors with multiple cores and our appliance can have 40 cores doing the processing right now, with more in the next version right around the corner.  Most home computers have 2-4 cores processing data traffic, with maybe 8 Gigabytes for RAM and maybe a 1Terabyte disk.

    Big data is really big, as are capabilities to process and use the data.

    •  Just went and verified - servers have 8TB of (4+ / 0-)
      Recommended by:
      JDWolverton, JML9999, StrayCat, Cliss

      RAM, not just 1TB.  Those are also relatively well-equipped, but when time is money, they really don't work out to be too expensive compared to how much data you wind up crunching through them.  When a Fortune 100 company has to find a piece of data - for example, when you visit a website and want to know what your account looks like or reconcile your statement from six months ago, you expect the company to be able to provide that in a couple of seconds.  That's what the companies demand from big data and that will continue to grow.  I'm not even going to address what a government might demand for performance.

      •  Link? (3+ / 0-)
        Recommended by:
        JML9999, StrayCat, Cliss

        I was looking into the RAM too. The 192 is a limitation of Windows O/S which I accepted because of relatability of most of my readers. I would think Lynx O/S would support Tb's of RAM.

        I don't have a contact at CDW anymore. When I spoke with a friend who does he mentioned their sales staff is under so much pressure these days, they don't have time to chat like they used to. The only way to stay up on the new equipment is to read hardware trade publications and I have enough trouble staying current on heath EHR to do that.

        Thanks for your input.

        If a nation expects to be ignorant and free, in a state of civilization, it expects what never has and never will be. Thomas Jefferson

        by JDWolverton on Tue Jul 02, 2013 at 10:22:35 AM PDT

        [ Parent ]

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site