Cent OS 7 fix yum against another client

The problem: I did some experiments on one of my clients, resulting on a client that is having a different package list than the other. I will call that client B. Let’s call the ‘healthy’ client A. When you make yum update, B is not able to get the latest kernel, since it is already quite altered. The goal is to make B to reach A without cloning the system or things like that. I managed in several steps.

rm -f /var/lib/rpm/__db*
rpm –rebuilddb
yum clean all
  • Create package list on A, copy it to B, install them on B
on A: rpm -qa > installed-software.log
on B: yum -y install $(cat installed-software.log)
  • Copy relevant configuration files from A to B.
on B: rsync -av A:/etc/pki* /etc/pki* --progress --delete-after
on B: rsync -av A:/etc/yum* /etc/yum* --progress --delete-after
on B: yum clean all; yum update
  • Regenerate grub so that it gets the new kernel.
on B: yum install grub2
on B: grub2-mkconfig --output=/boot/grub2/grub.cfg

The package information (pki) is the key here to have the same on A and B. I hope you don’t need to do this frequently, it’s really a waste of time. Be careful when you mess around, maybe it’s better to do it on a container or something like that!

Advertisements

A docker Hello World on Cent OS 7

Previously I gave you my recipe for containers on Cent OS 7. Now we are going to enable dockers and play a little with them. I’m following a version of this guide to dockers on Cent OS 7. Actually it’s pretty easy: first we install them.

yum install docker docker-ce docker-ce-edge docker-ce-test

This command on my machine able to run lxc already, is installing docker.x86_64 plus 15 dependencies. It could be some packages are also updated, so be aware. Then we start the daemon and call the hello-world container. This is my output.

root@machine ~ ## > systemctl start docker
root@machine ~ ## > docker run hello-world
Unable to find image 'hello-world:latest' locally
Trying to pull repository docker.io/library/hello-world ... 
latest: Pulling from docker.io/library/hello-world
ca4f61b1923c: Pull complete 
Digest: sha256:be0cd392e45be7997ec7f98e23905c

Hello from Docker!
This message shows that your installation 
appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" 
image from the Docker Hub. (amd64)
 3. The Docker daemon created a new container 
from that image which runs the
executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to 
the Docker client, which sent it to your terminal.

To try something more ambitious, you can run an Ubuntu container 
with:
 $ docker run -it ubuntu bash

If you do as suggested above,  a docker ubuntu container is pulled. It looks like this:

root@machine ~ ## > docker run -it ubuntu bash
Unable to find image 'ubuntu:latest' locally
Trying to pull repository docker.io/library/ubuntu ... 
latest: Pulling from docker.io/library/ubuntu
660c48dd555d: Pull complete 
4c7380416e78: Pull complete 
421e436b5f80: Pull complete 
e4ce6c3651b3: Pull complete 
be588e74bd34: Pull complete 
Digest: sha256:7c67a2206d3c0
root@b5e65d5f8ae0:/# ls

There are a lot of open questions to be answered. I will see what I can do with them.

EDIT: I run a Quantum State Diffusion docker on my test machine easily by typing this:

docker run  quantum_state_diffusion/Dockerfile  # just run it
docker run -it --entrypoint=/bin/bash tabakg/quantum_state_diffusion 
#get a shell on it

Now it’s time to write my own docker.

Python/WXpython Hello World

Welcome back to the fantastic world of computing. Today we’re going to learn the difference between  python and wxpython. First we need to be sure we have “some” python in our system. For that we need a shell (the black window, AKA terminal). There we type:

user@machine$ >which python
/usr/bin/python

If you get an output, you  have python. If not, you may need to install it as root, load a module, or ask your system administrator. Now we can create a folder called test, and there our first script, hello.py. For that we use our favorite text editor. The file will simply contain one line:

print("Hello world python")

Python is a script language. You can run python scripts in two ways, from the python command prompt “interactive mode”, or asking python to run it. We will first run it. To do it, on your shell, type:

$ > python hello.py

The expected output is to see printed Hello world python. You just run your first python script. Are you not getting excited? Now let’s do the same from the command prompt. To go to it, simply type python on your shell. The output should look similar to this

$ > python
Python 2.7.5 (default, Nov 6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" 
for more information.
>>>

Now after the >>> simply write  print(“Hello world python”). You should see under it, a Hello world python message printed, and again the prompt. To exit the interactive mode, type exit(). Do not forget the parenthesis, since “exit” is a function, not only a command.

But the cool thing of python is that you can easily create popping up windows for your script using one of the python libraries called wxpython. Let’s try it out as described here. We make a file called hello_wx.py with this in:

#!/usr/bin/env python
import wx
app = wx.App(False) # Create a new app
frame = wx.Frame(None, wx.ID_ANY, "Hello World") # A Frame 
frame.Show(True) # Show the frame.
app.MainLoop()

If we run this one by typing  python hello_wx.py we get a nice but useless window, that we can move around and close. It does nothing, but it’s a beginning… you can thank import wx  for this feature. And now you know from where (some of the) windows are coming from 😀 😀  … see you around!

 

C/C++ Hello World

Since this is my first article, I’m going to start without a long explanation. I will suppose you have a bash shell. You need to check that you have a compiler. As an user, you can type

user@machine$ > which gcc
/usr/local/bin/gcc  #output
user@machine$ > which g++
/usr/local/bin/g++  #output

If you don’t get any similar output (the path may vary) maybe you need to install a compiler (as root) , you need to load some module, or you need to ask your system administrator to install one compiler for you. Supposingly you managed to get one, we’re going to make a test folder (that I will call test) and there I’m going to create a new file with my favourite text editor. The choice on text editors is a personal one. You can use, in order of complexity and features, vi, vim, nano, emacs, gedit or geany. Or something else that I don’t use. I will only post the content of the file. You need to end up with something like this:

#include <stdio.h>
int main() {
printf("Hello world C\n");
return 0;
/*  this is a comment on C */
}

Let’s save this file as hello.c. Now we’re going to create another file with this content:

#include <iostream> // needed to use cout
int main() {
     std::cout << " Hello world C++ \n";
     // this is a comment on C++
}

And save this one as hello.cpp. You see? You just finished your first program on C, and your first program on C++. Now we need to compile it to run it! I will say you have gcc as a default compiler, since it’s quite common. To compile hello.c, you need to type on your shell:

user@machine$ > gcc hello.c

Now do ls to find the output of your compilation, called a.out. This is your first C program! To run it, do

user@machine$ > ./a.out

And as a result you should get “Hello world C” printed on your shell. It doesn’t do so much, but it’s a beginning. Let’s rename it, for example, to hello_c, and run it again. We go ahead and compile hello.cpp also. This is done by typing

user@machine$ > g++ hello.cpp

Now do ls to find the output of your compilation, called a.out. This is your first C++ program! To run it, do

user@machine$ > ./a.out

And as a result you should get “Hello world C++” printed on your shell. Now you can say you know how to write a “hello world” on C and C++. Congratulations! 😀 😀

Basic Hellos

I’m going to start from tomorrow on a serie of posts about how to start programming on a selection of the most popular languages. My very ambitious aim is to demonstrate that, once you know the grammar, all the languages are the same. Like for a human language, the most important thing is not what you have to say, but how you say it. That is, the algorithm.

A leginon CentOS 6 container on CentOS 7

Continuing with our cooking recipes, I’m going to install leginon on a CentOS 6 container running on CentOS 7. We start creating a CentOS 6 container:

lxc-create -n centos6 -t download -- -d centos -r 6 -a i386
lxc-start -n centos6

To log in via ssh we need first to install the openssh server. Also, let’s install more things that we will need if we want to have a nice leginon installation experience.

yum install -y git which openssh-server wget \
unzip tar ffmpeg ffmpeg-devel

Now we better restart the container and log in as root via ssh. We’re ready to start installing. Since we have CentOS 6 we can directly use the autoinstaller.  I’m not going to discuss the previous steps of the complete installation but go to step 6 directly. So we do

python centosAutoInstallation.py

This is retrieving and installing every package we need using different tools, so we need to monitor everything goes smoothly. In my case, the first time it didn’t finish successfully due to several packages missing. I will blame my basic container for that. I choose no sample download, no software install, and I get:

ERROR: Failed to run Myamiweb setup script.
You may try running http://localhost-something 
in your web browser.

Let’s try to go ahead and ignore the error. Next issue occurs when I try to run firefox from inside the container.

firefox
process 16203: D-Bus library appears to be incorrectly set up; 
failed to read machine uuid: 
Failed to open "/var/lib/dbus/machine-id": 
No such file or directory
See the manual page for dbus-uuidgen to correct this issue.
 D-Bus not built with -rdynamic so unable to print a backtrace
Redirecting call to abort() to mozalloc_abort
Segmentation fault

The solution to the D-Bus library problem is easy.

> dbus-uuidgen > /var/lib/dbus/machine-id

I open firefox and I check that I can see the leginon start page. This is not the end, now we need to do the database server installation.  I cut and copy from the HOWTO:

# yum install mysql mysql-server # already done by autoinstaller  
ls /usr/share/mysql/my* --> they are indeed there
cp -v /usr/share/mysql/my-huge.cnf /etc/my.cnf

I modify my.cnf as suggested, I add  default_storage_engine=MyISAM at the end of the file, and I service mysqld start, chkconfig mysqld on. So far so good.

Since we want to have a container as portable thing, instead of creating the database, we are going to import it. We do as I wrote on phpMyAdmin upload and fix old database plus an additional step:

chown -R mysql /var/lib/mysql/

Otherwise starting mysqld fails with the error:

[ERROR] Can't open and lock privilege tables: 
Table 'mysql.servers' doesn't exist

We can open phpMyAdmin on firefox now, or do the checks suggested on step 13 and 14. All looks clear. What now? We do have already a leginon server up and running on ‘real’ hardware , so instead of  runnig the online setup wizard by visiting http://yourhost/myamiweb/setup or http://localhost/myamiweb/setup to create the myami website’s config file, as described here, we simply rsync the folder from the running one to our container, and perform service httpd restart. Provided the data folder is mounted in the container, and so on, we now have a working leginon container. In principle we can stop here, but we want to update it regularly so that we have the latest software version. The first update I do manually. First we clone the beta version of myami like this:

git clone -b myami-beta http://emg.nysbc.org/git/myami myami-beta
cd /path/myami-beta
./pysetup.sh install

And we check that everything works:

~/myami-beta/leginon ## > python syscheck.py
 --------------------------------------------------------------
 Looking for previously installed Leginon...
 Leginon found here: /usr/lib/python2.6/site-packages/leginon
 *** It is best to uninstall your previous 
Leginon before installing the new one. 
The best way to uninstall is to move it to a backup
location, just in case you need to revert to the old version.
--------------------------------
 Python:
 Python executable (if wrong, check PATH in your environment):
 /usr/bin/python
 Python module search path (if wrong, check PYTHONPATH):
 /root/myami-beta/leginon
 /usr/local/protomo-2.4.1/lib/linux/x86-64
 /usr/local/EMAN/lib
 /usr/lib/python26.zip
 /usr/lib/python2.6
 /usr/lib/python2.6/plat-linux2
 /usr/lib/python2.6/lib-tk
 /usr/lib/python2.6/lib-old
 /usr/lib/python2.6/lib-dynload
 /usr/lib/python2.6/site-packages
 /usr/lib/python2.6/site-packages/PIL
 /usr/lib/python2.6/site-packages/
setuptools-0.6c11-py2.6.egg-info
 /usr/lib/python2.6/site-packages/wx-2.8-gtk2-unicode
 Python version: 2.6.6
 OK (at least 2.3.4 required)
 Python says home directory is: /root
 --------------------------------------------------------------
 Python Imaging Library (PIL):
 importing Image module...
 PIL version: 1.1.6
 OK (at least 1.1.4 required)
 -------------------------------------------------
 MySQL Python client (MySQLdb):
 importing MySQLdb module...
 Python MySQL version: 1.2.3c1
 OK (at least 1.2 required)
 -------------------------------------------------
 numpy:
 importing numpy module...
 numpy version: 1.4.1
 OK (at least 1.0 required)
 --------------------------------------------------
 scipy:
 importing scipy.optimize module...
 testing for leastsq function...
 OK
 -------------------------------------------------
 wxPython:
 importing wx module...
 wxPython version: 2.8.12.0
 OK (at least 2.5.2.8 required)
 Testing a wxPython application. 
Close the window that pops up...
 TEST
 wxPython test successful

So it seems to work. We could copy the stuff from myami-beta to the location given on the test,  /usr/lib/python2.6/site-packages/leginon. Time for the power user to test it! We’ll see how long until we screw it up 🙂

 

 

CentOS 7 error while loading shared libraries: libfftw3.so.3

Case description: on my centos7 container, I sync ProgramX from the container host (centos 7 also) and when I try to run it, I get the error.

ProgramX: error while loading shared libraries: 
libfftw3.so.3: cannot open shared object file: 
No such file or directory

To fix it it’s very easy.

yum install fftw3

That’s it. Just to add a little background information, fftw are the Fast Fourier Transform libraries. They may be present as a part of other installed programs, but in principle, they don’t come by default, since fast Fourier transforms are not common system operations 😀