The water wedding (XXIX)

I’m definitely not anymore completely myself at this moment, the colourful drinks and the information overflow is making me dizzy. After our arrival, magically fountains of alcohol popped out of the tables, and invisible servants placed everywhere big trays with exotic food. Or exotic for me. Giant Mussels with funny shapes. Transparent colourful little spheres, fish eggs I believe. Fried insects looking like still alive, with their open wings. Mushrooms, truffles, berries, and very delicate moon-shaped breads. Super tasty little sausages. I ate and drunk like there was no finale, while following Hakim as a cat does, looking for his approval but silent. Beautiful and smart, I hope.

Oddly enough the hall seemed to adjust itself on size according to the people coming. Now it’s as big as a football field, not that I play football, but I have the concept of that size in the way I know the Moon is not a eye in the sky that is staring at us, but a satellite. Fun fact, I seem to remember also that there were humans landing on it. I stare at one of the water spikes that act as trees on the field, partially covering a group of tables with a changing filtered sunlight. I think everything must be controlled by fields. The same fields that the Shadows are using to hide, maybe. Then, where’s the energy source? Is it the cascade itself? Yes, a waterfall can generate energy. But how they pipe it this way?

– Hakim. – He looks at me. He’s certainly sad. Maybe he was forced to live outside the Court, or maybe this celebration is bringing him back to his past. He sips from a gorgeous glass and smiles me back, waiting for my question. So I ask. – What are we going to do after this is over? – His face contorts with sadness first, then with rage, I believe.

– What do you mean after it’s over? After all that I’ve given you! Of course we’re going to come back and continue our lives. Or maybe you are not coming back… – his eyes suddenly show a dark light that I never saw before on hom. Or did I? – maybe you will choose one of these… puppets…instead of me. Do you have the guts to do something like that?

– Stop Hakim, you’re hurting me! – His grip relaxes somehow. He grabbed me at the end of his last sentence, and for a moment, I thought he was going to throw me to the floor. But he didn’t. People around us (isn’t that the Ministry of Cultural Affairs?) seem to show interest of how our fight is developing, and my lover seems to realize it.

– Sorry Kris. – He grabs another glass of something blueish, and in a second the liquid is gone down his throat. He suppresses a burp, but I see it. I know him. Or I think I know. He hesitates for a couple of seconds just to take after yet another drink, as I was expecting. – I think I’m drinking too much. I hope you will understand why later. And I’m not able to clean my system without help as before… so… will you come with me to help me to release some liquids? Or will you stay here with Mr. Important Ministry of Something? – The alluded ministry, that was following our fight with puppy eyes from the closest table, suddenly seems to be offended and leaves the table making some muffled sounds and murmuring. And I’m happy he’s gone.

– No, of course I accompany you. I wouldn’t be here without you, and you know it.

At that precise moment, instead of running one over the other, we need to turn our heads. The wedding procession starts to arrive to the hall, and the hall itself expands, it changes, tables sliding out of the central part that seems to be slowly growing up, devoid of people. We were, at this moment, two groups, one to the left, one to the right. In the middle, a wide water passage, like a low catwalk, is forming. We are very close to border of it, so good news is that I will manage to be very close to Him/Her, to our God, to our Leader, to our Emperor. I wonder if I will survive it.

Advertisements

The water wedding (XXVIII)

– I want to introduce you to the Court. – I look around while I’m being driven from where we stand to the closest cluster. Where are the guards? Are they all in Shadow mode? I wonder if I can detect them. I wonder how can I detect them.

– Hakim…are we safe here?

– Why do you ask? – His face looks funny to me for a few seconds. Was it an effect of the light or was it having some concerns about it? – You know they can hide. – He continues speaking, after the first odd reaction. – You know you will not know they are there, unless you scan them. And you will not do that, will you? – He winks while saying that. I know he refers to our enhanced abilities. Last time I remember he winked is when we discussed about the peasant girl across the road. I saw them together, and I thought he cheated me. But the wink here has no context. What does it mean here and now? – Yes, you’re a good boy. You will not break the protocol. I know you don’t. Even if you don’t know it. But basically, the people you see in this “room”, if I can call it this way, are the rulers of our world, and right now, this must be one of the safest places on Earth to be. – Hakim smiles, but I see his lips twisted a little, a sign I learned to understand as a sign that either he’s lying or he is not sure about what he says. I take the first.

– I will not question further. You told me you wanted to introduce me to some people?

I’m not sure I can distinguish one from the other. But we go from table to table introducing ourselves. I don’t know why we do this, but the other guests seem to enjoy our visit, and sometimes, we get pieces of intel. Like that the System is very active the last days, maybe they heard about the Wedding, Like the image of the Empire. I didn’t actually know we had so many: one image of it is that of father that tells you what you need to do to achieve your goals. The other one is of a failed scientific experiment.

The water wedding (XXVII)

– You didn’t expect it to be so magnificent, did you?

After leaving the Tailor’s house, it took us not so long to reach the bottom of the crater. At the bottom, a road was leading us to something that, from the distance, resembled a mirage, some kind of building apparently actively vibrating and shining. Like a figment of my imagination. We parked where we see other vehicles (is that a tank?) and continued by walk: Hakim seems to know what to do but he doesn’t care to explain it to me, he just gives orders. I cannot avoid frowning.

– The Ceremonial Hall at the Fountain is not always visible, due to its nature. It can’t be there all the time, all around the year. Reasons are obvious. – I smile. He sees me and continues with the history lesson. –  No, I don’t know who built it. And yes, I was here before. The last time, it was for the dead of the father of him/her… that was…fifty years ago, I believe. Gosh, time flies!

The Ceremonial Hall is built on water. The water, as it flows from the cascade into the crater, is being channeled by crystal pipes in a fashion that on its gravitational thrust, it creates intricately designed walls and rooms. The main wall, where we just entered, is looking like a roofless temple guarded by big columns, columns that are empty inside. Their size is enormous. And they must be to create the illusion. Delicate spikes fluctuate over the columns, water feeded by Archimedes principle or similar fluid laws. There is no door, just a missing wall, or a passageway, that we take in.

From inside it’s even more spectacular. What I see now, to the left and the right of the road, is a group of approximately a hundred people clustered in random groups around circular tables, the tables filled with all kind of bottles, and topped with fountains. The general sensation is that we are open air, but in a castle. Over the hall, or around it, beyond the columns, there are colourful grandstands. I don’t see them all: they appear blurry through the water curtain. Any single person in sight around has a characteristic feature: a ridiculously long tophat, a hairdo in the shape of a still fountain, or visible breasts. How about that! Is the man over there naked and painted in silver? And is that one blinking us, blinking me?

– No I didn’t expect it to be the half of this spectacular. And it’s so big and fragile… but at least you don’t have pirates! – Hakim seems to be amused by my expression.

– Just for the record, this is my first Wedding. – I know it is convenient to fill up some kind of memento to say you and your couple are together. I know that the best is to do it in front of a priest, or somebody else with similar power. Even so, live is (was) so brief, and we were (are?) so poor that the wedding is not usually on our list of important milestones.

The water wedding (XXVI)

The Tailor happened to be a very silent man. Or so I thought until I found out he was missing the tongue…I guess he lost it due to knowing too many secrets. Even so, he and Hakim seemed to be old friends, and our transaction was fast: we walked easily across the Tailor’s amazing workshop to his wardrobe and then we started to try some.

– You look simply gorgeous, my dear. – Hakim was already dressed, but I was hesitant about the model. I do not have cultural baggage, since I know the Empire is not my birthplace. The problem is, I know nothing else than the Empire’s culture. Hakim has chosen a chang kben on silver, grey and blue, topped with a dark green tuxedo over a black t-shirt. Now he’s fiddling with a box of skin colors, while carrying on his hand several hairpins and an extension for his bun. I have chosen a white and blue churidar, aquatic colors seem the appropriate choice for the event, but I have my doubts. How quickly can I pee with this? What if I need to run?

– C’mon, choose something! – My mate is now admiring the result of his painting work over one of the standing mirrors of the changing room. The mirror, edged by a golden frame, has an enormous crack on it to the right, like if somebody punched it. But Hakim doesn’t seem to care about. On the contrary, he seems to take pleasure observing how his arm seems to be detached and attached again to his body due to the crack.

– I told you you are beautiful, no matter what you dress. So please do it, I don’t want to meet his new customers – says him pointing to the Tailor. He doesn’t seem to take it. – Take this. – He hands me a set of a beautiful capri pants and a jacket. I grab them, enjoying the silk and golden tones. They seem to complement perfectly Hakim’s outfit on a weird way, like if we were the negative one of the other.

– Maybe I should have given up on searching for something. You definitely are the man of the couple! – I say. He gives me one of his brilliant smiles. Also the Tailor, which is weird.

– Go there and try it out! – He points to one of the spaces behind the colourful and ancient sandalwood folding screens, where we can sit and dress while the person outside can guess what we do inside. A changing room. The expression comes to me in a natural way, but it’s not a natural one. I doubt for the moment I have my nanos under control. But I have, they seem to tell me. I will try to believe it.

– You see? That’s the one! Now, we need to choose some makeup…

– No, no makeup for me, please. – Something is telling me the makeup is not going to be compulsory. – Just come here and comb my hair, as you do at home. Please?

Hakim comes to me and kisses me. The Tailor sees what’s going on and leaves us alone.

CryoSPARC 2 install on CentOS 7

There’s a new CryoSPARC available. We had a previous installation, but it can not be updated to this version. You need to ask for a new license. This is a boring post about how it is installed, plus the installation output, a little bit cropped so that this post is not eternal. Detailed instructions you can find by the company’s web. The installation, although mysterious sometimes, was pretty smooth. Kudos for the team! I installed it under the user account user. The bolds and the colors are mine.  Total installation time of a master + one slave is around one hour.

Installing the master

/install.sh --license $LICENSE_ID --hostname master
******* CRYOSPARC SYSTEM: MASTER INSTALLER ********************
Installation Settings:
License ID : XXXXXX
Root Directory : /xxx/user/cryosparc2/cryosparc2_master
Master Hostname : master
Database Directory : /xxx/user/cryosparc2/cryosparc2_database
Base HTTP Port Number : 39000
Developer Install : false
Operating System : Linux
Insecure Install : false
Version : v2.0.27
******************************************************************
The cryoSPARC System master installation will host a database, 
web application, and command layer for cryoSPARC, but will not 
perform processing jobs.
Are the above settings correct?
1) Yes
2) No
**************************************
Setting up hard-coded config.sh environment variables
**************************************
Installing all dependencies.
Dependencies have changed - reinstalling.
-----------------------------------------
Installing anaconda python...
----------------------------------------
PREFIX=/xxx/user/cryosparc2/cryosparc2_master/deps/anaconda
installing: python-2.7.14-h1571d57_29 ...
installation finished.
-----------------------------------------
Done.
anaconda python installation successful.
-----------------------------------------
deps bundle installation.
Processing ./deps_bundle/pip_pkgs/Flask-JSONRPC-0.3.1.tar.gz
Processing ./deps_bundle/pip_pkgs/Flask-PyMongo-0.5.1.tar.gz
Processing ./deps_bundle/pip_pkgs/libtiff-0.4.2.tar.gz
Requirement already satisfied: Flask>=0.10 in 
...some others here...
Requirement already satisfied: MarkupSafe in 
Skipping bdist_wheel for Flask-JSONRPC, 
due to binaries being disabled for it.
Skipping bdist_wheel for Flask-PyMongo, 
due to binaries being disabled for it.
Skipping bdist_wheel for libtiff, 
due to binaries being disabled for it.
Installing collected packages: 
Flask-JSONRPC, Flask-PyMongo, libtiff
Running setup.py install for Flask-JSONRPC ... done
Running setup.py install for Flask-PyMongo ... done
Running setup.py install for libtiff ... done
Successfully installed 
******* CRYOSPARC MASTER INSTALLATION COMPLETE ****************
CryoSPARC System master has now been installed. 
Add bin directory to your ~/.bashrc ?
1) Yes
2) No
#? 1
The following lines were appended to your ~/.bashrc:
export PATH=/xxx/user/cryosparc2/cryosparc2_master/bin:$PATH
Please re-start your terminal shell to make the cryosparcm
command available. 
Afterwards, you can start this master instance with:
$ cryosparcm start
To find out about other available commands, use:
$ cryosparcm help
In order to run processing jobs, you will also need to install
one or more cryoSPARC worker nodes and connect them to this
master node.

Let’s install the worker them, on the master itself.

Installing the worker

./install.sh --license $LICENSE_ID 
--cudapath /xxx/local/cuda-9.1/
******* CRYOSPARC SYSTEM: WORKER INSTALLER**************
Installation Settings:
License ID : XXXX
Root Directory : /xxx/user/cryosparc2/cryosparc2_worker
Version : v2.0.27
*****************************************************
CUDA check..
Found nvidia-smi at /usr/bin/nvidia-smi
CUDA Path was provided as /xxx/local/cuda-9.1/
Checking CUDA installation...
Found nvcc at /xxx/cuda-9.1//bin/nvcc
The above cuda installation will be used but 
can be changed later.
*****************************************************
Setting up hard-coded config.sh environment variables
*****************************************************
Installing all dependencies.
Dependencies have changed - reinstalling.
-----------------------------------------------
Installing anaconda python...
-----------------------------------------------
PREFIX=/xxx/user/cryosparc2/cryosparc2_worker/deps/anaconda
installing: python-2.7.14-h1571d57_29 ...
Python 2.7.14 :: Anaconda, Inc.
installation finished.
-------------------------------------
Done.
anaconda python installation successful.
---------------------------------------------
deps bundle installation.
...
Requirement already satisfied: MarkupSafe>=0.9.2 in
Skipping bdist_wheel for Flask-JSONRPC, 
due to binaries being disabled for it.
Skipping bdist_wheel for Flask-PyMongo, 
due to binaries being disabled for it.
Skipping bdist_wheel for libtiff, due to 
binaries being disabled for it.
Skipping bdist_wheel for pycuda, 
due to binaries being disabled for it.
Skipping bdist_wheel for scikit-cuda, 
due to binaries being disabled for it.
Installing collected packages: 
Flask-JSONRPC, Flask-PyMongo, libtiff, pycuda, scikit-cuda
Running setup.py install for Flask-JSONRPC ... done
Successfully installed 
Flask-JSONRPC-0.3.1 Flask-PyMongo-0.5.1 
libtiff-0.4.2 pycuda-2017.1.1 scikit-cuda-0.5.1
******* CRYOSPARC WORKER INSTALLATION COMPLETE ***************
In order to run processing jobs, you will need to connect this
worker to a cryoSPARC master.
*******************************************************

It is interesting that the worker seems to install its own python over the one of the master. But if it works, I don’t care. Now we need to connect master and slave installations. This is done with a one-liner comand.

Connecting master and worker

bin/cryosparcw connect --worker master 
--master master --ssdpath /xxx/user/cryosparc_ssd
-----------------------------------------------
CRYOSPARC CONNECT -----------------------------
-----------------------------------------------
Attempting to register worker master to command master:39002
Connecting as unix user user
Will register using ssh string: user@master
If this is incorrect, you should re-run this 
command with the flag --sshstr <ssh string> 
---------------------------------
Connected to master.
Current connected workers:
---------------------------------
Autodetecting available GPUs...
Detected 8 CUDA devices.
id pci-bus name
---------------------------------------
0 0000:02:00.0 GeForce GTX 1080
...the others here...
8 0000:83:00.0 GeForce GTX 1080
---------------------------------------
All devices will be enabled now. 
This can be changed later using --update
----------------------------------------
Worker will be registered with SSD cache location 
/xxx/user/cryosparc_ssd 
----------------------------------------------
Autodetecting the amount of RAM available...
This machine has XXX RAM .
-------------------------------------
Registering worker...Done.
You can now launch jobs :-)
--------------------------------------
Final configuration for master
lane : default
name : master
title : Worker node master
resource_slots : {u'GPU': [0, 1, 2], u'RAM': [0, 1,.., 35]}
hostname : master
worker_bin_path : 
/home/user/cryosparc2/cryosparc2_worker/bin/cryosparcw
cache_path : /home/user/cryosparc_ssd
cache_quota_mb : None
resource_fixed : {u'SSD': True}
cache_reserve_mb : 10000
type : node
ssh_str : user@master
desc : None
-----------------------------------------

To add a new node, I log in as use on the computer and run the command bin/cryosparcw connect. In this way I quickly added 4 nodes. I didn’t need to recompile it or anything, since my user ‘user‘ is a network user and our sytems are all CentOS.  I wonder if I’ll see it running any time soon. And I wonder which marvels it will produce.

Creating a CentOS 7 systemctl service

I don’t know why I’m doing this. Really. But I may want to do it again in the future. Yes, sometimes I read myself and I’m ashamed. Today I’ve learned all by myself how to create a system task. You can learn how to use systemd here. It’s not so complicated, but nothing is if you know how to do it, right?

My question, specifically was: how I create a service for a shell script so I can start it and stop it like a daemon? I’ve created a slightly more complicated shell script that the one on the previous link. I call it bitservice. that looks like this:

#!/bin/sh
### BEGIN INIT INFO
# Description: Multi-purpose bit service
### END INIT INFO

SCRIPT=`/root/bin/bit_configure`
RUNAS=root

PIDFILE=/var/run/bitdaemom.pid
LOGFILE=/var/log/bitdaemon.log

start() {
if [ -f /var/run/$PIDNAME ] && 
   kill -0 $(cat /var/run/$PIDNAME); then
   echo 'Service already running' >&2
   return 1
fi
echo 'Starting service\u2026' >&2
local CMD="$SCRIPT &> \"$LOGFILE\" & echo \$!"
su -c "$CMD" $RUNAS > "$PIDFILE"
echo 'Service started' >&2
}

stop() {
if [ ! -f "$PIDFILE" ] || ! kill -0 $(cat "$PIDFILE"); then
   echo 'Service not running' >&2
   return 1
fi
echo 'Stopping service\u2026' >&2
kill -15 $(cat "$PIDFILE") && rm -f "$PIDFILE"
echo 'Service stopped' >&2
}

case "$1" in
   start)
   start
   ;;
   stop)
   stop
   ;;
   restart)
   stop
   start
   ;;
*)
echo "Usage: $0 {start|stop|restart}"
esac

Let’s explain it. Basically, the bitservice can start, stop or restart what is described on SCRIPT. I don’t give systemd arguments but the command on SCRIPT can be easily changed. So it can be a multi-purpose service. There are different levels of complexity. For example, you may want to use only two files to define your service. Or maybe you want to give some arguments to your service. I used 3 files.

  1. The one that runs on SCRIPT (called  bit_configure), the script itself
  2. The bitservice,  the service for the scrip (start/stop/etc) that I place on /root/bin/
  3. The system entry, on /lib/systemd/system/ , that I call bitservice.service

Tha last file, my bitservice.service is describing how the system will operate bitservice.

[Unit]
Description=BIT service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
PIDFile=/var/run/bitservice.pid
ExecStart=/root/bin/bitservice
PrivateTmp=true
Restart=always
RestartSec=180s
StandardOutput=null
StandardError=journal

[Install]
WantedBy=multi-user.target

What  you see above is that biservice.service calls /root/bin/bitservice each 180s. Logs are supposed to be minimal. We control them through the StandardOutput. There is no configuration file, but we can do that. Or we can rewrite bitservice it to take secondary arguments. In any case, we may want to redirect the service output, or to change the restart frequency. And then, apply the changes. For that we simply type:

systemctl daemon-reload
systemctl start bitservice.service

But after all this work, anyway, I decided to use a crontab instead. The reason is the verbosity level of the system daemon. It is quite complex, and you may not get the systemd messages the place you want them. Or you may need to fiddle up with dmesg, or even worse, with journalctl. That will end up affecting all your system. Anyway, at least I can say I know how to write a service! Ah, in case you are fancy about continuing with this,  here you have an example of a php service for you to try it out 😉

Windows robocopy configuration

I’m kind of puzzled about how different Linux rsync and Windows robocopy are. Maybe if you are a Windows user you are familiar with them, but I’m not. On StackOverflow there is a list of options to apply to robocopy, but none of them satisfy me. Here you have a comprehensive list of options. I quote:

/L :: List only - don't copy, timestamp or delete any files.
/X :: report all eXtra files, not just those selected.
/V :: produce Verbose output, showing skipped files.
/TS :: include source file Time Stamps in the output.
/FP :: include Full Pathname of files in the output.
/BYTES :: Print sizes as bytes.

/NS :: No Size - don't log file sizes.
/NC :: No Class - don't log file classes.
/NFL :: No File List - don't log file names.
/NDL :: No Directory List - don't log directory names.

/NP :: No Progress - don't display percentage copied.
/ETA :: show Estimated Time of Arrival of copied files.

/LOG:file :: output status to LOG file (overwrite existing log).
/LOG+:file :: output status to LOG file (append to existing log).

/UNILOG:file :: output status to LOG file as UNICODE 
(overwrite existing log).
/UNILOG+:file :: output status to LOG file as UNICODE 
(append to existing log).

/TEE :: output to console window, as well as the log file.

/NJH :: No Job Header.
/NJS :: No Job Summary.

/UNICODE :: output status as UNICODE.

Now what is the meaning of the output, in case you enable one? What are those “Tweaked”, “Lonely”, and “Extra” ? They refer to RoboCopy “Classes” of files:

File        Exists In   Exists In        Source/Dest     Source/Dest   Source/Dest
Class       Source      Destination      File Times      File Sizes    Attributes
=========== =========== ================ =============== ============= ============
Lonely      Yes         No               n/a             n/a           n/a
Tweaked     Yes         Yes              Equal           Equal         Different
Same        Yes         Yes              Equal           Equal         Equal
Changed     Yes         Yes              Equal           Different     n/a
Newer       Yes         Yes              Source > Dest   n/a           n/a
Older       Yes         Yes              Source < Dest   n/a           n/a
Extra       No          Yes              n/a             n/a           n/a
Mismatched  Yes (file)  Yes (directory)  n/a             n/a           n/a

The original post from Superuser is here. If you want, here it is the official Microsoft documentation, that I didn’t find very useful. I feel like I’m now the Robocopy King!