I previously used plotty to get a python plot on a website. It’s time to elaborate. Now we want to have two plots with data from a CSV file. It’s not as complicated as it sounds. Allow me to show and explain.
import dash from dash import dcc, html from dash.dependencies import Input, Output import plotly.express as px from plotly.subplots import make_subplots import plotly.graph_objects as go import pandas as pd
app = dash.Dash(__name__)
df = pd.read_csv('data3colums.csv') ##columns are named here and you need to use these names df.columns=['Time','data','Size(T)'] fig = px.line(df, x="Time",y="data",markers=True) fig2 = px.line(df, x="Time",y="Size(T)",markers=True) app.layout = html.Div(children=[ # All elements from the top of the page html.Div([ html.H1(children='Time series one'), html.Div(children='''Title one'''), dcc.Graph(id='graph1',figure=fig), ]), # New Div for the new 'row' of the page html.Div([ html.H1(children='Time Series two'), html.Div(children='''Title two'''), dcc.Graph(id='graph2',figure=fig2), ]), ]) if __name__ == '__main__': app.run_server(debug=True,host='IP-OF-MY-MACHINE')
EDIT: this post has been backdated to “yesterday” because I forgot to publish it, so another post may come later. Hopefully…this will mean I had time for my things 😉
Just a note. I’m kind of busy with a hundred other things before my Easter holidays so I don’t have time to edit my drafts. Today I was faced again with the next: while archiving an old user’s data, I found 15T tar balls and bigger. I ask the users to tar the projects they don’t expect to need in the next months, and some people simply run the command over the thing and wait until it’s done 😂. But that is not the best way, anyhow, here I am to fix it. Our backup system is working with tapes, so I decided to split the file onto smaller, easier to handle pieces with split. Out there you can find a lot of ways to solve this problem. But I found what I wanted on this article from networkworld.
split --verbose -b 250G BIG.tar.gz BIG.
At the end I get my files named BIG.aa, BIG.ab, etc. Still I need to wait quite some time for the split to happen. And still I need to send the chunks to a tape. But now I have a better overview of what’s going on, since I can tell the size of the project by the number of balls 😉.
Well as you know I’m playing with GPFS on my cluster, I’ve created a new filesystem and now I wanted to have it acknowledged by a second cluster. I was following the procedure on the IBM Specturm Scale 5.0.0 Admin guide, from where I took this table (page 347):
To summarize, there’s cluster owningCluster (the one where I configured the new filesystem ptemp) and accessingCluster (the other one). In my setup, where the error was produced, accessingCluster is already sharing a filesystem with owningCluster from a third cluster that I will call STORAGE. Actually there are even more clusters in the picture, but then, if your setup is simpler my solution may work. What I did:
Copy owningCluster keys to a new folder on the accessingCluster login node
At accessingCluster, run mmauth add owningCluster -k id_rsa.pub
At accessingCluster, run mmauth grant owningClasster -f ptemp
At accessingCluster, run mmremotecluser add owningCluster
At accessingCluster, run mmremotefs add ptemp -f ptemp -C owningCluster -T /ptemp
Att owningCluster, run mmauth grant accessingCluster -f ptemp
If I try now to mmount ptemp on accessingCluster, I get the above error. Something like this:
## >systemctl status gpfs gpfs.service - General Parallel File System Loaded: loaded (/usr/lib/systemd/system/gpfs.service; enabled; vendor preset: disabled) Active: active (running) since XXX CET; YYY ago Main PID: 111 (runmmfs) Tasks: 389 Memory: 16.0G CGroup: /system.slice/gpfs.service - 222 /usr/lpp/mmfs/bin/mmksh /usr/lpp/mmfs/bin/runmmfs - 333 /usr/lpp/mmfs/bin/mmfsd
[E] The key used by the contact node named quorum1 <inteface> has changed. Contact the administrator to obtain the new key and register it using "mmauth update". [E] The key used by the contact node named quorum2 <inteface> has changed. Contact the administrator to obtain the new key and register it using "mmauth update". [E] The key used by the contact node named quorum3 <inteface> has changed. Contact the administrator to obtain the new key and register it using "mmauth update". [E] The cluster named STORAGE has not authorized this cluster to mount file systems. Contact the cluster administrator and request access. [E] A node join was rejected. This could be due to incompatible daemon versions, failure to find the node in the configuration database, or no configuration manager found. : Error=MMFS_PHOENIX, ID=0xAB429E38, Tag=3848153: R eason code 669 Failure Reason [E] A node join was rejected. This could be due to incompatible daemon versions, failure to find the node in the configuration
This is not inmediate but the whole owningClusterdegraded over time. Which is bad, of course, because after the join of ptemp I went home and then I started to get complains from the users. To solve this we need to go to the three clusters in this game. We can check the public keys on /var/mmfs/ssl to see if they are the same and run mmauth show to list the clusters. A sample output looks like this.
mmauth show Cluster name: STORAGE Cipher list: AUTHONLY SHA digest: long-key-number-one File system access: (none authorized)
Let’s go and fix this, now that we have the picture. On owningCluster we deny and delete the accessingCluster (the one that created the mess). Something like this:
## >mmauth deny accessingCluster -f all mmauth: Propagating the cluster configuration data to all affected nodes. mmauth: Command successfully completed ## >mmauth delete accessingCluster mmauth: Propagating the cluster configuration data to all affected nodes. mmauth: Command successfully completed ## >mmauth show Cluster name: owningCluster (this cluster) Cipher list: AUTHONLY SHA digest: long-key-number-two File system access: (all rw)
Now we go to STORAGE, maybe to quorum1, and on it, deny and delete the owningCluster, copy the keys from owningCluster to a specifc folder (not the one with the current keys!) and rejoin it. Like this:
## > mmauth deny owningCluster-f all mmauth: Warning: the new access control does not apply on remote nodes that have the specified file system currently mounted. The new access control will be applied on the next mount of the file system. mmauth: Propagating the cluster configuration data to all affected nodes. mmauth: Command successfully completed ## > mmauth delete owningCluster mmauth: Propagating the cluster configuration data to all affected nodes. mmauth: Command successfully completed ## > scp * root@owningClusternode:/var/mmfs/ssl/* /root/newkeys/ ## > cd /root/newkeys/ ## > mmauth add owningCluster -k id_rsa.pub mmauth: Propagating the cluster configuration data to all affected nodes. mmauth: Command successfully completed ## > mmauth grant owningCluster -f GPFSBASE mmauth: Granting cluster owningCluster access to file system GPFSBASE: access type rw; root credentials will not be remapped. mmauth: Propagating the cluster configuration data to all affected nodes. mmauth: Command successfully completed
Almost done. We go to the @owningClusternode, and rejoin only STORAGE, that has GPFSBASE. We have copied the STORAGE keys to a new folder. In a few lines:
## >mmauth add STORAGE -k id_rsa.pub mmauth: Propagating the cluster configuration data to all affected nodes. mmauth: Command successfully completed
I think that’s it Anyway, this mess I don’t deserve to any of my enemies! Sooo…many… system administrators 😁😁😁…. BONUS : GPFS remote filesystem will not mount.
Well well well. We want to read and write as fast as possible, but how can be sure that our client is able to perform properly? I have added 2 new disks, one native, single, U2, and another, GPFS mounted, composed by different disks of different technologies (M2.U2, SATA SSD). I know, mixing is bad. But this is how I live now! The aim is to get the same speed on both, if possible, of course. I start by mounting the U2 on /data, and run ddrescue on it like this:
On /data we end up with a block file called “zero” but I suppose it’s fine, since the disk is new. I translate the above results: my U2 speed is 1.5 GB/s. We then mount the GPFS share on /cache and do the same:
As you can see, a dissapointing 200 MB/s. 😩😩😩. So time to play around with the GPFS configuration, we’ll see how far I go…I will let you know if you are interested 😉.
Scenario: the user comes saying that he cannot get his X11 windows exported anymore on the destination cluster login node after the crash of one linux client he was using. Crash meaning, of course, an accidental switching off of the client performing the ssh to the login node. So I sit and test with my user account from a couple of clients to the same destination login node, successfully getting my xclock popping up after the ssh. Inspired by this post from Kittycool timeout in locking authority file, as sudo on the destination cluster I remove the .Xauthority from the user’s home and ask him to try ssh again, still not managing to get the graphics exported.
As you may know, the xauth controls the authorisation to get the graphics. It can be refreshed, it can be recreated, but also it can be also tampered by manually adding the magic cookie to it, as described also here and in my previous post. But I can’t seem to be able to add the cookie to the user’s xauth. The permissions, after comparing them with mine, also look fine. I even move the .Xauth to another location and copy mine (that works) with of course the right user’s rights, and move his whole home folder to user_bkup just to create a new clean home folder that we could use. All of that unsuccessful.
Then I try mkdir and I get a message. The answer to the riddle was always on this post. There was no free inode quota for the user. How come? Well, on my behalf, just to say that I didn’t know the user was experimenting with chopping his data onto 2 million small files for a reason that I can’t understand, and I’m in principle not responsible of the cluster, only yet another super user. Errare humanum est, or something like that.
Bits and Dragons 