Linux command tips - updating

how to view the process priority

ps -lef  --- l means long mode
you can view the priority and nice value

S   UID   PID  PPID  C PRI  NI   RSS    SZ WCHAN  TTY          TIME CMD
S     0     1     0  0  80   0  1540  4839 poll_s ?        00:00:01 init
S     0     2     0  0  80   0     0     0 kthrea ?        00:00:00 kthreadd
S     0     3     2  0 -40   -     0     0 migrat ?        00:00:00 migration/0
S     0     4     2  0  80   0     0     0 ksofti ?        00:00:00 ksoftirqd/0

how to list only just the first level of directories in a directory

1. find . -maxdepth -type d  (it will include the directory itself)
2. ls -ld */

how to use BASH to read a file line by line

[root@X001 tmp]# while read line
> do
> echo $line
> done < test.txt
this is a test file from rafa
the file is very good ha ha
could you give me the file here
[root@X001 tmp]# cat test.txt
this is a test file from rafa
the file is very good ha ha
could you give me the file here

linux hash command:

Linux uses hash table to speed up the commands finding process. when linux found a command, it stores it in the hash table so next time when it needs to check the command, it can get from hash table. the cost of looking via hash table is O(1)

linux command line {} expand:

when we use { } in bash, it will be expanded by bash.

for example: 

mkdir -pv /var/tmp/{a/b,c} = mkdir -pv /var/tmp/a/b /var/tmp/c

How to recover a file deleted by mistake when it is still opened in the process:

Find the file by ls -l /proc/<pid>/fd/, then copy it by cp /proc/<pid>/fd/<fd> ~/myfile.txt 

MySQL user and privilege management

privileges management

MySQL provides privileges that apply in different contexts and at different levels of operation:

·  Administrative  privileges enable users to manage operation of the MySQL server. These privileges are global because they are not specific to a particular database.
·  Database privileges apply to a database and to all objects within it. These privileges can be granted for specific databases, or globally so that they apply to all databases.

Grant or revoke privileges

>grant [priviledge] on object to ‘user’@’host’

>revoke [priviledge] on object from ‘user’@’host’

Object can be

• ON *.*
• ON db_name.*
• ON db_name.table_name
• ON db_name.routine_name

Flush the privilege so it will take effect immediately

>flush privileges

 

Common Administrative privileges:

CREATE USER	create drop or rename other users
PROCESS	display of information about the threads executing within the server
RELOAD	use flush and reset
REPLICATION CLIENT	enables the use of SHOW MASTER STATUS and SHOW SLAVE STATUS
REPLICATION SLAVE	accounts that are used by slave servers to connect to the current server as their master
SHOW DATABASES	see the databases
SHUTDOWN	shutdown the server
SUPER	miscellaneous administrative operations
ALL [PRIVILEGES]	shorthand. It stands for “all privileges available at a given privilege level” (except GRANT OPTION).
USAGE	no privileges

Common database privileges:

alter	modify the table
alter routine	modify the store procedure
create	create database and table
create routine	create store procedure
create view	create the view
delete	do delete SQL
drop	drop db or tables
execute	execute the store procedure
grant option	grant privileges to other accounts
index	modify index
insert	do insert SQL
select	do select SQL
update	do update SQL

MySQL user management

The user is identified by userid  + host, two users with same userid but different host are considered as different user. The host field indicates where the user is logged from. The user information is stored in mysql.user table

Create user:

Users can be created by the “create user” command

>create user ‘user’@’host’ identified by ‘password’

Host can be ip address, localhost or %. % means every host. sample command:

>create user 'rafax'@'%' identified by '123456';

show online users

>show processlist

The above command can see the online users and their activities, but you may need privilege to see other users, otherwise you will only see your own activities.

Rename user

User name or host can be renamed by the following command:

>rename  user 'user1'@'host1'  to 'user2'@'host2';

Delete the user

>drop user [user]@[host]

For example:

>drop user ‘rafax’@’%’

BASH LOOPS

Loop is to repeat some code continuously and exit based on the some expression evaluation.

FOR LOOP

basic for loop struct:

for statement

do

       commands block

done

there are quite a few common examples

• for variable in 1 2 3 4 5
• for variable in {1..5}
• for i in $(seq 1 2 100)
• for day in Monday Tuesday Wednesday Thursday Friday Saturday Sunday
• for argument in "$*"
• for(( integer = 1; integer <= 5; integer++ ))
• for(( a=1, b=5; a <= 5; a++, b-- ))

WHILE LOOP

while expression

do

       commands block

done

the statement will be evaluated if true then will do the block, and re-evaluate the expression. The loop will exit untile the expression is false.
unitl loop is quite similar but until will always execute the loop at least once then evaluate the expression

break exit the loop

continue ignore the left loop block and continue an new loop

SELECT LOOP

select is very similar to the case statement in bash, it provides an interactive user interface.

Here is the sample code for user to select from a specified list

echo "What is your favorite color? "

select color in "red" "blue" "green" "white" "black"

do

    if [ $color ] ; then

              echo "You have selected $color"

              break

       else

              echo "Invalid selection"

       fi

done

it will ask the user to key in a

such as

what is your favorite color?

1)      red

2)      blue

3)      green

4)      white

5)      black

#? Red

Invalid selection

#? 0

Invalid selection

#? 1

You have selected red

Python - Dictionary

Dictionary is a very important build-in data structure in Python. It is an order-less key-value data store.

Dictionary creation

Dictionary can be created by assignment as below:

mydict = { ‘key1’:’value1’, ‘key2’:’value2’,’key3’:’value3’}

key and value are separated by : , key-value pairs are separated by ;

dictionary can also be created vi comprehension.

mydict = { id:str(id) for id in xrange(10)}

Note, dictionary is order-less, we can’t assume it is stored by some order, in fact, it is stored by key’s hash value.

Dictionary access

Dictionary’s value can be access by it’s key as

Value ---- mydict[key].

But if the key is not in the dictionary, it will generate an Exception.

Dictionary’s value can be reset by assignment

mydict[key]=value, if key is not in the dictionary, it will create an new entry for it.

Dictionary element delete

The key-value pair can be deleted as

del (mydict[key])

Dictionary’s key-value pair can be traversed by for in statement

for key in mydict:

       print mydict[key]

it can be also traversed by items() function which will return the key-value list.

for (key,value) in mydict.items():

       print key, value

the traversal of dictionary can also be achived by iteritems(), iterkeys(), itervalues(), they are the iterators.

Some other common functions for dictionary

keys() and values()

the function will return a list of keys and values. Note the order can’t be assumed.

get()

When you access the dictionary by mydict[key], if the key is not in the dictionary, it will raise exception, a safer way is to use

dict.get[key,default]

It will return the value of the key or default if key not existing

update()

the dictionary can update by another dictionary.

mydict1.update(mydict2)

if mydict2’s key in mydict1, mydict1’s value will be replace by mydict2

if mydict2’s key not in mydict1, mydict1’s key-value pair will be inserted.

Dictionary’s copy

There are two types of copy

• Shallow copy: it will copy by reference or pointer
• Deep copy: it will copy the exact value

In other words, when you do shallow copy from a to b, when you changed the value in a, b’s value may be changed as well, but if it is a deep copy, the value won’t be impacted.

import copy

mydict = {‘a’:[1,2,3,4]}

shallowdict = copy.copy(mydict)

deepdict = copy.deepcopy(mydict)

shallowdict now is {'a': [1, 2, 3, 4]}

deepdict   now is {'a': [1, 2, 3, 4]}

deepdict['a'][0]=0   --- set the deep copy dictionary value

deepdict    now is {'a': [0, 2, 3, 4]}

mydict   not changed {'a': [1, 2, 3, 4]}

shallowdict['a'][0]=0  -- set the shallow copy dictionary value

mydict  changed {'a': [0, 2, 3, 4]}

Sort the dictionary

We may sort the dictionary by sort and lambda

#Sort by key

sort(dict.items(), key=lambda d:d[0])

#Sort by value

sort(dict.items(), key=lambda d:d[1])

HTML 5 page layout enhancement

New Input type

HTML 5 provides some handy input types and it can save the developer lots of time and efforts.You may find the detailed input types here

http://www.w3schools.com/html/html5_form_input_types.asp

but be aware the new HTML feartures may not be fully supported by individual browsers. Use it carefully when design your pages.

Here are a summary:

• color:                                  user to choose the color
• date/time/datetime/datetime-local       user to choose the date and time
• email                                   contain an e-mail address.
• week/month                              choose week or month of the year
• number                                  input a number
• range                                   an number from a specific range
• search                                  input a search text
• tel                                     an telephone number
• url                                     a URL address.

A HTML page should usually organised as below

Header tags can be used any where of the HTML body. You can user id attribute to refer to it.

It can be used mutliple times.

Nav tag is used for navigation section. It is recommanded to use one nav tag in the whole page.

Usually the tag is combined with ul (unordered list)

Section tag is used for different sections. eg, in a main page, we may have sports, news, games sections.

Article tag is used for any specific articles.

Aside tag is used for some content aside from the content it is placed in.

Footer tag is used for footer but it can be in wholepage, section, article

Git tab management

Git provides a snapshot function called tag. usually tag is used to refer to some milestone version.

Make a tag

#git tab –a tagname –m “comment”

eg: #git tag –a V1.0 –m “this is the version 1.0”

make an tag for previous commit

#git tab –a tagname –m “comment” [commit id]

Show how many tags in the current branch

#show tag

v1.0

v2.0

Delete the tag

#git tag –d tagname

If you want to delete the tag in Git server (eg github), we need to push it to remote server

#git push origin :refs/tags/[tagname]

Checkout the tag

Same as checkout the branch

#git checkout tagname

Push the tage to remote Git server

Push one tag

#git push origin gitname

Push all tags

#git push origin --tags 

Git branch management

Branch management is very important especially when git project is shared by team. Project can set branches, when a programmer works on the branch, it won’t interrupt others work. When the sub-tasks finished, the programmer can merge the branch so that it can be visible to others.

Usually the project should have a default branch called master as below

    

create an branch

command:

git branch test          # create an branch called test
git checkout test        # switch to the branch called test

or:

git checkout –b test    # create a test branch and switch to it.

now git repository is like:

 modify the test branch and commit it

merge the master and test branch

git checkout master       #you need switch to the branch you want to merge
git merge test            #merge the branch now

now it is like:

delete the branch if not used any more

commands:      git branch –d test

a short summary of basic branch management commands:

• check the branch status            #git branch
• create an branch                   #git branch bname
• switch to a branch：               #git checkout bname
• create and switch to the branch：  #git checkout -b bname
• merge the branch to current branch：#git merge bname
• delete the branch：                 #git branch -d bname

branch conflicts management

when the two branches are both made changes, they can’t be merged directly.

You will have to manually fix the conflicts then merge the file

there are some practical rules for branch management

1. project has a stable and release branch called master
2. project should have other sub-main branches as test, dev, uat
3. team members should create their own branches from test, dev. They should not create branch from master directly
4. team members should merge their codes to dev or test
5. master can only be merged from one single branch, eg dev.