20TB fileserver (1 Viewer)

[Lehmden]

AW: 20TB fileserver

Hello.
My NAS is a lot less spectacular than yours but does it's job perfect. I have around 13 TB of .mkv and .avi laying around. I prefer 1080P if possible as on my 42" LED TV I can see clearly the difference between 1080P and 720P. A good 720P is a lot better than an ugly 1080P, of course. But a good 1080P is way more fun than a good 720P...

The hardware I use is simple. A poor AMD Sempron 140, a Biostar. Board with OnBoard GPU and GBit Lan, 2 GB DDR2 Ram and Windows 7 as OS does it really well. A cheap Midi Tower with 10 HDD internal, a 350 Watts PSU from BeQuiet, that's all I need. My Server has to handle MySQL, Apache,... also, but no MediaPortal TV Server. I prefer to watch TV without the need of running two PCs and 12 HDDs at the same time... Power consumption is way enough while watching stored Movies at all. Next step forward will be via USB3.0 external HDDs as the room inside the case is filled completely with HDD's.

No Raid, no Hot Spare or something like that. I have used this all before but this simple solution does the best I've ever had. Data Pool is created with junctions so I have complete control what file is where but also have a single place to access my data. Looks cool if you have a folder containing 10.000GB Movie Files on a 350GB partition...

More complex solutions has too many fault possibilities... Dying Raid Controllers I've seen al lot more than dying HDDs. Especially short time after a compatible spare controller is no longer available any more. This Worst Case Scenario (total loss of all data from the raid array due to a defect controller without possibility to buy a new compatible one) has happened 3 times during my computer carrier now. This won't happen again, I swear. No more Raid Controllers, never again... My actual setup is running a lot longer now without any issues than any other solution I've tried before.

The tip, keep the server as it is, this is the key to success. No extra Software if possible, only absolute necessary updates, and so on. I agree this 100%...

btw.. my Server is reachable 24/7 but did not run 24/7. If it's not needed it goes to sleep (S4) and wake up automatically if needed. Saves lot of power, noise, hardware and last but most important, lots of money (300€++ a year).

 

[MrBobC]

I'll also chime in, as one day I should like to make a thread about my storage, but I have 35 1.5TB HDDs in 7 different 5-disk sets of RAID-5 through Windows Server 2003 SP1. Soon I shall have to get the next 5-disk set as I am getting full. I know they're s/w RAID, but I've had no problems in terms of losing data (except one time which was my own fault). I chose s/w because h/w RAID solutions = $$$, not to say 35 HDDs do not = $$$. They are in two servers, one of which is a Norco 4220. They are housed in a cabinet I made myself with parts from Lowe's (got the idea from Milhouse on these forums), and it's 24U, but soon I will be getting an actual 42U rack. It houses my personal PC which I run cords to from the server room, a VCR for encoding VHS, Server 1, Server 2, an HTPC which powers the projector in my office (run cords), and I hope to someday have a TV PC in there as well. I am interested in this FREENAS, as I understand it is more stable than S2003? I shall have to look more up about that. I am in the process of making mkvs out of my isos. I was creating hardlinks on the server so TVSeries could see all the episodes, but I have decided mkvs will be a better way to go and will save space in the long run. Also, there are 3 additional PCs in other rooms of the house connected by wire to the 16-port switch I have. A PC for every TV, I suppose. I shall have to take pics one of these days.

 

[poyntzj]

I have taken advantage of the HP Microserver cashback and bought two of these
Each are just over 200 GBP + VAT, so say 240 GBP. 100 GBP cashback 140-GBP each. And much faster than the old NAS units I had
I have mine with 4GB of RAM, but you can go 6, 4 x 2TB disks in a software RAID 5 under windows (the original disk is in the Optical drive bay as boot, but I know people who are running FreeNAS off a USB).

A routine checks the state of the RAID each hour and if there is a problem I am emailed. I also get a daily check
Also SW raid as if a controller dies, you need to get the same. With SW, you can move the disks to another box and import them - which I have done in the past

I went down the Windows route as I have a few windows tools that used to run on an old machine and are now on these boxes so allowing me to retire another old bit of kit

Power usage is low as the disks do spin down (my UPS can do a deep battery test for 30 mins and then give up - not due to battery being too low, just because it clearly got bored)

Works well as when I am watching something in the front room, the other unit is there, but the disks are spun down, reducing electricity costs.

Cheers

 

[Brudertac]

AW: 20TB fileserver

poyntzj

How is the Read/Write Performance with SW Raid5?

Thanks!

 

[superted125]

Re: AW: 20TB fileserver

@poyntzj

How is the Read/Write Performance with SW Raid5?

Thanks!

Just to add my 2c

I just built a 8x 2TB ZFS Raidz2 (Raid 6) fileserver and my benchmarks gave me 380MB/s write, 790MB/s read from the array (bytes not bits). And it also happily maxed out gigabit (havent yet setup my link aggregation for dual gigabit goodness.

Anyway, i was very pleasantly surprised by the results

 

[Jay_UK]

Hi there,

No one in their right mind should run a large disk volume with no RAID/resiliency - The risk of a component/disk failure is just too high.

I will leave the debate of H/W versus S/W RAID to you guys, both are a suitable solution - depending on your needs.

If we are talking internal RAID, all I will say, if you get a decent H/W RAID card, the manufacturer normally maintains "cross card" support, where you can simply fit a newer/replacement card from the manufacturer and it will still work with your RAID set. The only times you have issues, is where you've used a cheap/inbuilt type RAID solution, where the manufacturer doesnt ensure continuity with their products.

J. (over 20 years in IT infrastructure)

 

[cfforce]

Im running a ZFS server (Openindiana / Napp-it) with 2x(7x2TB in Raidz2).

TANK used 6.24T
TANK available 11.4T

So total of 17,6TB

Using Samsung 2TB Eco drives, with 1 HS and 1 120Gb SSD Drive as cache.
Also Teamed up 3x1Gb network cards, for my iSCSI booting Clients (diskless), and PC's
Seq-Write: 389MB/s
Seq-Read: 563MB/s

The System is running ESXi 4.1, with the virtualized Openindiana and TVserver (MP) (DVBS2 Passthru)

 

[AlexanderIanHall]

For anyone looking at building a NAS server, whether with the HP Micro Servers, Norco 20 HDD beasts or something in between then highly recommend taking a look at UnRaid (Home) or FreeNas rather than take the Microsoft route. The linux based NAS builds let you re-use low spec hardware and are more than capable of handling 45 GB Blu-Ray ISO images over a gigabit ethernet network via DaemonTools Lite through to MediaPortal.

I personally use UnRaid, it handles resilience without needing to use RAID 5, therefore you get more effective disk usage (space wise) and it can grow as your needs increase.

Plenty of other options out there as well, I can only personally comment on UnRaid and Synology NAS systems as that is what I have. What one is best for you depends on your current and future storage needs and how 'dirty' you want to get in making your own solution.

hth

Alex

 

[Nosferax]

My new HTPC setup is as follow:

• Dell XPS 8500 "Special Edition" (i7, 16gig ram, HD7870 GPU, 256Gig SSD, 2Tb HDD, 1x BD-r, 1x LG BD burner)
• 2x ProBox 4 drives USB 3.0/eSata ( 4x 2Tb HDD each ) - via USB 3.0
• 1x ProBox 4 drives USB 2.0/eSata ( 4x 1Tb HDD ) - via eSata
• 1x Hauppauge Colossus HD capture card

I use FlexRaid for safety on each box, so the 4th drive in each box is used for parity. This leaves me with 15 Tb of space. I also did a bit of cleaning in my media collection. I got rid of the old XVid/DivX encodes that I had kept since I've started in this hobby in 2003. I also flushed the iso of disc that I have on physical media in my collection since it has been confirmed to me that MP can manage offline collection.

From now on, I only rip DVD to "vanilla" MKV (uncompressed) with english/french audio and subs with MakeMKV. For BluRay I use either DVDFab to make physical copy or I use RipBot264 for a x264 video re-encode ( CQ=18, Auto-Crop, either 720p or 1080p) with Flac audio. For the TV capture I use WinTV 7 to schedule and record and VideoRedo to cut out the commercials. I then use Handbrake to recompress them ( CQ=18, Auto-Crop, either 1080/720/480 ) into a MKV.

The PC is way overkill for a strict HTPC but I'm planning to do some gaming on it. I also use it to rip and transcode and also record on it, so the extra power is welcome.

 

[maiky]

It´s awesome with much storage, just threw out my old file server (8x1TB RAID5) and switched out for 6x4TB RAID5 on "real" adaptec raid controller. Much fun