perl: warning: Setting locale failed on Debian GNU/Linux 11 (bullseye)

I love these titles. They look like coming from a disaster movie or something. I’ve encountered this one while trying some of our software on a Debian 11 docker. The full error looks like this:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = "en_US:en",
LC_ALL = "en_US.UTF-8",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

The solution is pretty easy. It’s described in this post. I choose apt-get install iperf. I log out and log in again and the error is gone. Yeah, this was more like the tip of the day, but …let’s go!

Advertisement

A web dashboard to monitor GPU usage : gpuview

Sample dasboard. Image taken from here.

The title says it all. I was looking for a way to check, in a glance, the status of multiple GPU server in my local network. Gpuview (avaliable here) turned out to be a quite good solution. It does lack trends, that I will need if I want to find out which GPUs are most used, but on the other hand it’s quite easy to install and configure. This is what I did on one GPU client to start it up:

## > yum install python3-devel
## > pip3 install gpuview
WARNING: Running pip install with root privileges 
is generally not a good idea. Try `pip3 install --user` 
instead.
Collecting gpuview
Using cached a4/gpuview-0.4.0.tar.gz
Collecting gpustat>=0.5.0 (from gpuview)
Using cached b/gpustat-1.0.0.tar.gz
Collecting bottle>=0.12.14 (from gpuview)
Using cached /bottle-0.12.23-py3-none-any.whl
Requirement already satisfied: six>=1.7
Requirement already satisfied: nvidia-ml-py<=11.495.46
Collecting psutil>=5.6.0 (from gpustat>=0.5.0->gpuview)
Using cached 0/psutil-5.9.2.tar.gz
Collecting blessed>=1.17.1 (from gpustat>=0.5.0->gpuview)
Using cached b/blessed-1.19.1-py2.py3-none-any.whl
Collecting wcwidth>=0.1.4 
(from blessed>=1.17.1->gpustat>=0.5.0->gpuview)
Using cached e5/wcwidth-0.2.5-py2.py3-none-any.whl
Installing collected packages: psutil, wcwidth, blessed, gpustat, bottle, gpuview
Running setup.py install for psutil ... done
Running setup.py install for gpustat ... done
Running setup.py install for gpuview ... done
Successfully installed blessed-1.19.1 bottle-0.12.23 
gpustat-1.0.0 gpuview-0.4.0 psutil-5.9.2 wcwidth-0.2.5
## > which gpuview
/usr/local/bin/gpuview

As you can see, after my install, I get a binary. In principle I can already run it. Which I do inmediately. The output in my host 1.2.3.4 looks like this:

## > gpuview run --host 1.2.3.4 --port 9988 --safe-zone 
Bottle v0.12.23 server starting up (using WSGIRefServer())... 
Listening on http://1.2.3.4:9988/ 
Hit Ctrl-C to quit. 

There are no registered hosts! Use `gpuview add` first. 

If I now open my browser pointing to http://1.2.3.4:9988/ I do get my first green box and details about the GPU running on 1.2.3.4. Now it comes the difficult part: to add some more hosts. On the documentation it is written that I need to add a new host like this:

$ gpuview add --url <IP:PORT> --name

But I somehow don’t manage to make it to work after launching gpuview in the second host. Error on the main host 1.2.3.4 reads:

Error: <urlopen error unknown url type: 1.2.3.5> 
getting gpustat from 1.2.3.5:9988

The new host appears if listed. Sample output (again with mocked up IPs)

## > gpuview hosts
# Name        URL
01. gpufive   1.2.3.5:9988
02. gpusix    1.2.3.6:9988

What is going on? Let’s remove the host and add it in another way. Like this (for gpufive)

 ## > gpuview remove --url 1.2.3.4:9988
 ## > gpuview add --url http://1.2.3.4:9988 --name gpufive

Now, we start again gpuview in host gpufive and our web dasboard on 1.2.3.4:9988 will start showing the gpus of the added hosts. Mission accomplished! For the moment…

Installing python3 on macOS 12.6 Monterey

Yeah I don’t know why I’m doing this, I rarely use python on my mac. I guess it’s just in case. I have followed this guide. I do have brew installed, but just in case, I will copy you the procedure.

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Now we install python with brew:

$ brew install python

And that’s it. Note that it took quite some time to complete in my 2017 macbook pro, but maybe because the install triggered quite some brew updates. Please be patient 😉.

Internal Server Error after update : munin not available

Sorry I don’t write here as frequently as before but I’ve been busy. The scenario: we are forced to update everything to the latest kernel and to clean all the extra packages. I don’t do it so when I’m back to my server and restart my munin service on the web side I get this message

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

I’m the server administrator, so I need to contact myself 😂😂😂. First of course I restart all the services. (httpd, munin, munin-node). That has no effect, so stop the munin service, clean all the logs and start them again. The log shows errors but nothing strange:

/var/log/munin ## > tailf munin-update.log 
DATE [INFO] Reaping Munin::Master::UpdateWorker
<XX>. Exit value/signal: 0/0
DATE [ERROR] In RRD: Error updating XXX
GPU_TEMP-g.rrd: conversion of 'N/A' 
to float not complete: tail 'N/A'

How about the httpd service itself? Let’s have a look:

/var/www/html ## > tailf /var/log/httpd/error_log
</p>
[DATE] munin-cgi-html: 
Can't open /var/log/munin/munin-cgi-html.log 
(Permission denied) at /usr/share/perl5/vendor_perl/Log/Log4perl/Appender/File.pm 
line 103.
<h1>Software error:</h1>
<pre>Can't open /var/log/munin/munin-cgi-html.log 
(Permission denied) at /usr/share/perl5/vendor_perl/Log/Log4perl/Appender/File.pm 
line 103.
</pre>
<p>
For help, please send mail to this site's webmaster, 
giving this error message 
and the time and date of the error.
</p>

Again, oddly enough, the message recommends to contact myself 😁😁. That I do, of course. Is there an easy solution for this? Reinstall? Move to a docker version? No, it’s much easier. I stop all the munin services and clean the log. Then I do:

/var/log ## > chown apache:munin /var/log/munin/

and systemctl restart munin. And my munin monitoring is back. Fir the records,

/var/log ## > ls -lh
...
rwx------ 2 munge munge 4096 DATE munge/
drwxr-x--- 2 apache munin 4096 DATE munin/

Time for a break. A summer break? not yet 😩😩😩…

Change the vi editor default color scheme

This turned out to be easier than I thought. I found the solution on this post. Step by step:

1. cd ~
2. vi .vimrc

Press the “i” key to enter insert mode. Write the next.

syntax on
colorscheme desert

Type :wq to save and quit the file. Then open the vi again. Your desert colorscheme should be there. It’s amazing I didn’t try this out before. This means I still have sooo much to learn! 😔😔

HOWTO : compile relion on a clean CentOS 7.X no modules available

It’s not obvious.There are a lot of pieces that need to match, a lot of prerequosites. Let’s say you have CUDA installed (not only the drivers) and you “yum” openmpi openmpi-devel gcc gcc-c++. Our shell needs to see them also, so please add to your bash_profile:

export PATH=/usr/lib64/openmpi/bin:/usr/local/cuda/bin/:$PATH

Next is just to follow the tutorial.

git clone https://github.com/3dem/relion.git
cd relion; git pull; mkdir build; cd build

The “simple” cmake will not work. We need to tell it all that we want. My cmake looks like this:

cmake -DAMDFFTW=ON -DCUDA_ARCH=86 -DCUDA=ON -DCudaTexture=ON -DFORCE_OWN_FLTK=ON -DCMAKE_CXX_COMPILER=g++ -DCMAKE_C_COMPILER=gcc -DMPI_C_COMPILER=/usr/lib64/openmpi/bin/mpicc -DMPI_C_LIBRARIES=/usr/lib64/openmpi/lib/ -DMPI_C_INCLUDE_PATH=/usr/include/openmpi-x86_64/ ..

make -j 8

If you have, for example, the wrong CUDA, you may get a compilation error pointing to your cuda installation (default /usr/local/). Just try with another one. If you have time 😉.

HOWTO : find a value in a bash array

Yeah a philosophical question almost. Why do you want to use a bash array? I have a lot of bash scripts, and for a man with a hammer every problem looks like a nail. In this Stackoverflow post I found my solution. That I will simply copy because, you know, it’s my (b)log.

if [[ " ${array[*]} " =~ " ${value} " ]]; then
    # whatever you want to do when array contains value
else 
    # whatever you want to do when array doesn't contain value
fi

Yeah I know. But it’s better than nothing, right? 🧐

MYSQL tips & tricks – output format and where clausules

By default when you run a mysl query you get a table as an output, with its header and so on. This is not wanted if we want to get the output stored in a bash variable, for example. Here you have the StackOVerflow post over the formatting issue. Or maybe you want to display all the results without the table? How about only the result? Let’s give some examples.

useremail=$(mysql --silent --batch -N -u $USERNAME -p$PASSWORD 
-D $DATABASE -e "SELECT email FROM mytable WHERE 
concept= \"$myconcept\" ");

The above command, on the proper bash script, will print me the email colum values from the table mytable of those entries with colum concept matching the value $myconcept. A sample output, for two matches, will look like this:

pepito@domain.org
jorgito@domain.org

Of course, this all will depend on what is stored on mytable. You see where am I going, right? 😉😉. Just in case, here I refresh you how to use the Mysql where with examples. PD: Yeah, I cleaning my notes. Finally 😔.

MySQL shell-script management & notes

I’m a shell guy, maybe because I miss the sea 😁😁. So whatever I can script I script. Now that I work with databases I’m scripting the data checks, access and similar. All big project starts with google always, and here you have my curated results:

MySQL: Run Query from Bash Script or Linux Command Line. A good start. The angular stone will be this line:

$ mysql -u USER -pPASSWORD -D DATABASE -e "SQL_QUERY"

where everything in capital letters can be a shell variable. Let’s make it more secure.

How to check if a table exists in a MySQL database? As usual, with an IF block. Adapted from that post, in pseudocode, the shell iffs would look like this:

### Check if DATABASE exists
if [[ $(mysql -u $USER -p$PASSWORD -e "$SQL_EXISTS" $DATABASE) ]] 
then
   echo " Table exists ..."
### Check if DATABASE has records 
if [[ $(mysql -u $USER -p$PASSWORD -e "$SQL_IS_EMPTY" $DATABASE) ]]
then
  echo " Table has records ..."

And here another StackOverflow post on the same line. Now we know we have a database or a table. Do we know if an element exist on it? Howtogeek can tell us how. From the post I will keep these lines that we can use to check if the element with id=1 has the specific value. I copy it:

field=$(mysql -u root -BNe 'USE test; 
SELECT label FROM test WHERE id=1')
if [ $field == 'TEST' ]; then
   //do stuff
fi

Go to the post for more details. Wrapping up, let’s check the whole table for an element, also in pseudocode…

ENTRY="mysearchedvalue";
QUERY="USE "$DATABASE"; SELECT columname FROM "$TABLE;
COLUMN=$(mysql -u $USERNAME -p$PASSWORD -BNe "$QUERY")
for field in $COLUMN; do
if [ $field == $ENTRY ]; then
    echo " Entry " $field " found"
else
   echo " Entry " $field " missing..."
fi
done

The above loop can probably be done in another hundred ways, but I think you get the message. If not check this post : loop through MySQL in bash. Happy coding, have a nice weekend! ❤️❤️

Quine – the ouroboros of GitHib in a docker

The uroboros. From the GitHub repository.

These things are better posted when hot. I have subscribed to the CodePen newsletters – as well to another dozen of newsletters – and in the last one I found this jewel. Damn, I may as well make a T-shirt out of it! In brief, we need, from time to time, automatic code generators (remember, no-code is also code). What if we could, somehow, generate the code to generate the code with the very code generator? Yeah, right? Wild. Here you have it. I have tested the docker on my MacBook, where I have docker desktop. Open a terminal, and type:

~$ > git clone https://github.com/mame/quine-relay.git
~$ > cd quine-relay
quine-relay $ > docker build -t qr .

quine-relay $ > docker run --privileged 
--rm -e CI=true -v 
$(pwd):/usr/local/share/quine-relay 
-v /usr/local/share/quine-relay/vendor qr
#######################
##  1: Ruby -> Rust  ##
#######################
ruby QR.rb > QR.rs
########################
##  2: Rust -> Scala  ##
########################

... a lot of stuff here that I cut ...

rc QR.rc > QR.rexx
#########################
## 128: REXX -> Ruby ##
#########################
rexx ./QR.rexx > QR2.rb
#############
## CHECK ##
#############
diff -s QR.rb QR2.rb
Files QR.rb and QR2.rb are identical

It takes a while ( around 1 hour to compile them all in my case) but also to draw the Sixtine Chapel. It is nice and colourful, and relaxing to watch. If you are my kind of person, of course 😁. At the end, I test that indeed the files QR.rb and QR2.rb are identical, despite of the fact it’s supposed to be the last action the docker does before dying. What did I achieve with this? Now I know I can translate my code, if needed to 128 other languages! 😁😁.