I feel like an stupid when I run a process like du on our big storage. It takes year and I can’t wait until it’s done. Therefore, I googled for help. You have half a dozen of tools available that will provide you with plots, but the easiest, more flexible solution I found, seems to be up to now, GNU parallel. With GNU parallel,
ls -d /home/* | parallel du -hs
goes from taking 9 hours to only 2. I know it’s not the best, but it’s a beginning. In this way, I get the info I want on a file, and I can mine the file and display its results on a web page. As I was doing before with a smaller storage. To install GNU parallel, I followed this guide. But basically:
wget http://ftpmirror.gnu.org/parallel/parallel-20170422.tar.bz2 bzip2 -dc parallel-20170422.tar.bz2 | tar xvf - cd parallel-20170422 ./configure && make && make instal
The installation goes smoothly, independent of the system. I tried on CentOS 7 (which has already a parallel yum package), CentOS 6, and RH7, without errors. First, of course, sometimes you need to install wget. Note that once installed, if you try to run it without a process, you get the next funny output:
parallel ...bla bla bla... parallel: Warning: Input is read from the terminal. parallel: Warning: You either know what you are doing parallel: Warning: (in which case: YOU ARE AWESOME!)
It’s nice to hear some compliments from time to time. Speaking about time, it’s time to work in parallel!