Tuesday, 26 August 2014
Saturday, 23 August 2014
The differnce between TCP and UDP
Notes to the blog:
http://www.javacodegeeks.com/2014/07/9-differences-between-tcp-and-udp-protocol-java-network-interview-question.html
This question what is the difference between TCP and UDP is
often asked during the interviews. It is also important for sysadmin and
program to understand for dairy work. Then we will explain what is the major
differences we can see between the two protocols.
Connection-oriented vs Connectionless:
this is the essential differnce between TCP and UDP. The
differnce also explains the other different points in the blog. when we say TCP
is connection-oriented, it means the protocol itself controls and manages the
connection session. It needs so-call 3-ways handshake and 4-ways termination.
TCP also has many control packages such as SYN,ACK to for flow control.
Meanwhile, UDP does not provide a 'connection' session for communication
parties.
Reliability:
TCP is reliable as the application can assume TCP provides a
reliable communication while UDP does not provide any delivery guaruntee
Ordering:
TCP also guarantees order of message as it can get the
sequence number from all the packages. while UDP does not care about the
package sequence.
Data Boundary:
TCP does not preserve data boundary, UDP does.In TCP, data
is sent as a byte stream, and no distinguishing indications are transmitted to
signal message (segment) boundaries. On UDP, Packets are sent individually and
are checked for integrity only if they arrived. Packets have definite
boundaries which are honoured upon receipt, meaning a read operation at the
receiver socket will yield an entire message as it was originally sent.
Speed:
In one word, TCP is slow and UDP is fast.
Heavy weight vs Light weight:
Because of the overhead mentioned above, Transmission
control protocol is considered as heavy weight as compared to light weight UDP
protocol. TCP header is longer than UDP headers.
Congestion or Flow control:
TCP does Flow Control. TCP requires three packets to set up
a socket connection, before any user data can be sent. TCP handles reliability
and congestion control. On the other hand, UDP does not have an option for flow
control.
Different Applications:
because TCP and UDP provides different behaviour or
characters, different application use either TCP or UDP as their transmission
protocal. for example, HTTP and FTP, they use TCP while SIP, TFTP use UDP.
Wednesday, 20 August 2014
some tips about MySQL query Optimization
1. use the index: for every SQL, use the explain statement to verify the execution plan.
2. dont use index for computing substatement
3. fine the search critieria as much as possible
4. create hash index on fields for min/max/count
5. do manual calculation on some nequalities
2. dont use index for computing substatement
3. fine the search critieria as much as possible
4. create hash index on fields for min/max/count
5. do manual calculation on some nequalities
Thursday, 7 August 2014
Python STL - OS
OS module.
os is a very important standard library (STL) in python for
system admin, sysadmin can use it to
interact with OS.
Common os functions
#list the directory’s content
>os.listdir(“pathname”)
#call shell script or
external binary
>os.system(“commands”)
>>> to be continue
Tuesday, 5 August 2014
very good summary about linux system administrator interview questions
this is I prepared for the interview for Linux admin/System Engineer Technical questions:
1)
how to install Digital
Certificates on apache/nginx
2)
Apache MPM types
3)
Linux symlink vs hardlink
4)
Linux run level (RHEL/Ubuntu)
5)
Linux static library link vs
dynamic library link
6)
How to install software from
source code and the meaning of each steps
7)
Files in /boot and their
function
8)
The content in /etc/grub.conf
9)
Linux booting procedure
10)
MYSQL database creation and
deletion
11)
MySQL primary key, unique key,
foreign key
12)
MySQL b+ tree and hash index
13)
MySQL index advantage and
disadvantage, how to create keys
14)
Select statement, left join,
right join, inner join
15)
How to shutdown a linux box
16)
DHCP requests procedure
17)
What is a proxy and reverse
proxy
18)
ASymmetric and Symmetric
encryption method
19)
How to use ssh key to login to
the system without a password (what’s the advantage)
20)
Some remote login security
rules
21)
How to execute shell commands
in MySQL
22)
Name some options of MySQLadmin
23)
What GUI tools you have used
for MySQL admin
24)
How to generate a service for
Linux
25)
RAID 0, 1, 5
26)
Name the method to copy a file
from a linux server to another server
27)
Rsync working mode. How to
setup an rsync environment
28)
Linux – how to enter the single
user mode
29)
How kill works in linux, name a
few common Signals in LINUX
30)
Describe the memory layout of a
procedure
31)
Ext2 special file permission
and ACL
32)
HDD layout? How about SSD
33)
What is the function of VFS
34)
List common Linux environment
variables
35)
How does CDN work
36)
How to forbid the root user to
login directly
37)
How to grant/revoke permissions
to MySQL user
38)
How to change mysql user’s
password
39)
OSI mode and TCPIP mode
40)
What is VLAN and it’s function
41)
Common HTTP method
42)
Common HTTP response code
43)
How to setup MySQL replication
44)
What is file’s magic number
45)
Linux file types
46)
Linux super lock content
47)
Linux inode content
48)
How ping works
49)
How tracert works
50)
DNS server type, records type
51)
How DNS query works
52)
TCP establishment and
termination method
53)
TCP flags
54)
What is TCP half close
55)
TCP 2MSL function
56)
List the TCP timers
57)
The format of a standard URL
58)
How can you design an Web
Server
59)
Web Server’s keepalive
advantage and disadvantage
60)
The responsibility of a system
administrator
61)
The difference between
procedure and thread
62)
Procedure life cycle
63)
The priority of the process,
nice value and RT
64)
How to track down a process
65)
How does strace/ltrace work
66)
Linux cache/buffer
67)
/etc/passwd, /etc/shadow
difference
68)
How to lock a user
69)
How to add a disk to linux
system and use it in production environment
70)
How to create file system (for
example ext4)
71)
Explain the output of top
command and the meaning of each field
72)
The difference between router
and switch, what is layer 3 switch
73)
how to make the OS secure
74)
myISAM vs InnoDB
75)
what is ACID
76)
what is the isolation level
77)
explain the backup and restore
for MySQL
78)
how to create user in mysql
79)
mysql log explaination
80)
explain what is PKI
81)
SSL establishment procedure
82)
List the file system Linux
supports
83)
List the linux disk blocks
84)
What is init process and it’s
function
85)
Explain the disk block types in
linux
86)
LVS working method and common
scheduler
87)
Linux should at least contains
which file partitions
88)
How to add modules in Apache
89)
How to integrate Apache and
Tomcat
90)
What’s the function of firewall
91)
NTFS vs FAT
92)
Hsrp vs vrrp
93)
Mysqldump common options
94)
Mysqlbinlog
95)
Common redundant method for
network
96)
Apache’s tuning
97)
Windows build-in group and
users
98)
File system common commands
99)
Yum configuration file and
repo.d sample
100)
The process status and flag
101)
The baseline for linux
performance (4 sub system)
102)
Linux performance checking
tools
103)
iptables tables and chains
104)
collion domain and broadcast
domain
105)
NAT types
106)
IP private sections
107)
how OSPF is working
108)
swap related commands. How to
setup swap partitions on the fly
109)
which software Apache rely on?
110)
How Apache and Nginx configure
a virtual host
Wednesday, 25 June 2014
ss (socket statistics) introduction
Almost every sys-admin know the powerful tool called ‘netstat’ but it is time to say bye-bye to netstat as we have a more powerful and concise tool called ‘ss’.
yum install iproute iproute-doc
while netstat is part of net-tools which has not been updated since 2001
when we compare the performance between ss and netstat, we can get the conclusion that ss is much faster than netstat, that is because ss uses TCP’s tcp_diag to get the statistics from linux kernel.
See when a linux handle 10000 TCP connections, netstat took over 1 second to get the output and only 0.01 second to get the ss status
5. ss -p #list the program associated with port
1. ss introduction
ss is the abbreviation for socket statistics. It is part of the iproute tool. If you can’t use ss, you need to install iproute such asyum install iproute iproute-doc
while netstat is part of net-tools which has not been updated since 2001
when we compare the performance between ss and netstat, we can get the conclusion that ss is much faster than netstat, that is because ss uses TCP’s tcp_diag to get the statistics from linux kernel.
See when a linux handle 10000 TCP connections, netstat took over 1 second to get the output and only 0.01 second to get the ss status
[root@X003 bin]# time netstat -ant > /dev/null
real 0m1.334s
user 0m0.230s
sys 0m1.101s
[root@X003 bin]# time ss -s
Total: 10439 (kernel 10442)
TCP: 11073 (estab 9203, closed 1, orphaned 800, synrecv 0, timewait 0/0), ports 10007
Transport Total IP IPv6
* 10442 - -
RAW 0 0 0
UDP 8 5 3
TCP 11072 10008 1064
INET 11080 10013 1067
FRAG 0 0 0
real 0m0.014s
user 0m0.000s
sys 0m0.012s
ss
2. Basic usage:
1. ss -s #Get basic statistics information
[root@X003 bin]# ss -s
Total: 180 (kernel 197)
TCP: 15 (estab 3, closed 1, orphaned 0, synrecv 0, timewait 0/0), ports 7
Transport Total IP IPv6
* 197 - -
RAW 0 0 0
UDP 8 5 3
TCP 14 8 6
INET 22 13 9
FRAG 0 0 0
[root@X003 bin]# ss -l
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 :::sunrpc :::*
LISTEN 0 128 *:sunrpc *:*
LISTEN 0 128 :::http :::*
LISTEN 0 128 :::ssh :::*
LISTEN 0 128 *:ssh *:*
LISTEN 0 128 127.0.0.1:ipp *:*
LISTEN 0 128 ::1:ipp :::*
LISTEN 0 100 ::1:smtp :::*
LISTEN 0 100 127.0.0.1:smtp *:*
LISTEN 0 128 :::55878 :::*
LISTEN 0 128 *:38186 *:*
[root@X003 bin]# ss -t
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 52 192.168.56.113:ssh 192.168.56.1:65433
ESTAB 0 0 192.168.56.113:ssh 192.168.56.1:65417
ESTAB 0 0 192.168.56.113:ssh 192.168.56.1:65418
5. ss -p #list the program associated with port
[root@X003 bin]# ss -p
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 52 192.168.56.113:ssh 192.168.56.1:65433 users:(("sshd",1789,3))
ESTAB 0 0 192.168.56.113:ssh 192.168.56.1:65417 users:(("sshd",1734,3))
ESTAB 0 0 192.168.56.113:ssh 192.168.56.1:65418 users:(("sshd",1759,3))
[root@X003 bin]# ss -n
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 52 192.168.56.113:22 192.168.56.1:65433
ESTAB 0 0 192.168.56.113:22 192.168.56.1:65417
ESTAB 0 0 192.168.56.113:22 192.168.56.1:65418
[root@X003 bin]# ss -e
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 52 192.168.56.113:ssh 192.168.56.1:65433 timer:(on,354ms,0) ino:74064 sk:ffff88002c8d5440
ESTAB 0 0 192.168.56.113:ssh 192.168.56.1:65417 timer:(keepalive,76min,0) ino:73782 sk:ffff8800299e4d80
ESTAB 0 0 192.168.56.113:ssh 192.168.56.1:65418 timer:(keepalive,76min,0) ino:73915 sk:ffff8800299e4700
Tuesday, 24 June 2014
Powerful DNS query tool - DIG
When we talk about the DNS query tool, we usually refer to nslookup. Now in this blog, we are going to talk about a more powerful tool called dig.
1.dig. Command without any parameters
2.basic command: dig @DNS domain-name record-type
3.common options: Here is the most common options we will need to keep in mind.
-t [type]: by default, it is A record(address) but you can set MX for query
1.dig. Command without any parameters
root@ip-172-31-7-84:~# dig
; <<>> DiG 9.9.5-3-Ubuntu <<>>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25338
;; flags: qr rd ra; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;. IN NS
;; ANSWER SECTION:
. 19664 IN NS d.root-servers.net.
. 19664 IN NS e.root-servers.net.
. 19664 IN NS f.root-servers.net.
. 19664 IN NS g.root-servers.net.
. 19664 IN NS h.root-servers.net.
. 19664 IN NS i.root-servers.net.
. 19664 IN NS j.root-servers.net.
. 19664 IN NS k.root-servers.net.
. 19664 IN NS l.root-servers.net.
. 19664 IN NS m.root-servers.net.
. 19664 IN NS a.root-servers.net.
. 19664 IN NS b.root-servers.net.
. 19664 IN NS c.root-servers.net.
;; Query time: 6 msec
;; SERVER: 172.31.0.2#53(172.31.0.2)
;; WHEN: Wed Jun 25 15:27:35 EST 2014
;; MSG SIZE rcvd: 239
2.basic command: dig @DNS domain-name record-type
root@ip-172-31-7-84:~# dig @8.8.8.8 rms.nsw.gov.au AAAA
; <<>> DiG 9.9.5-3-Ubuntu <<>> @8.8.8.8 rms.nsw.gov.au AAAA
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23205
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;rms.nsw.gov.au. IN AAAA
;; AUTHORITY SECTION:
rms.nsw.gov.au. 1799 IN SOA dns2.rta.nsw.gov.au. root.dns2.rta.nsw.gov.au. 2013120601 7200 3600 3600000 7200
;; Query time: 289 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Jun 25 15:29:14 EST 2014
;; MSG SIZE rcvd: 93
3.common options: Here is the most common options we will need to keep in mind.
-t [type]: by default, it is A record(address) but you can set MX for query
root@ip-172-31-7-84:~# dig -t MX rms.nsw.gov.au
; <<>> DiG 9.9.5-3-Ubuntu <<>> -t MX rms.nsw.gov.au
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 41964
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;rms.nsw.gov.au. IN MX
;; ANSWER SECTION:
rms.nsw.gov.au. 7200 IN MX 30 mx22out.rta.nsw.gov.au.
rms.nsw.gov.au. 7200 IN MX 40 waterways-b1.maritime.nsw.gov.au.
rms.nsw.gov.au. 7200 IN MX 10 cluster4.us.messagelabs.com.
rms.nsw.gov.au. 7200 IN MX 20 cluster4a.us.messagelabs.com.
rms.nsw.gov.au. 7200 IN MX 30 mx11out.rta.nsw.gov.au.
rms.nsw.gov.au. 7200 IN MX 30 mx12out.rta.nsw.gov.au.
rms.nsw.gov.au. 7200 IN MX 30 mx21out.rta.nsw.gov.au.
;; Query time: 14 msec
;; SERVER: 172.31.0.2#53(172.31.0.2)
;; WHEN: Wed Jun 25 15:31:10 EST 2014
;; MSG SIZE rcvd: 250
root@ip-172-31-7-84:~# dig -q www.google.com
; <<>> DiG 9.9.5-3-Ubuntu <<>> -q www.google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60498
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.com. IN A
;; ANSWER SECTION:
www.google.com. 166 IN A 74.125.237.180
www.google.com. 166 IN A 74.125.237.176
www.google.com. 166 IN A 74.125.237.177
www.google.com. 166 IN A 74.125.237.178
www.google.com. 166 IN A 74.125.237.179
;; Query time: 6 msec
;; SERVER: 172.31.0.2#53(172.31.0.2)
;; WHEN: Wed Jun 25 15:33:07 EST 2014
;; MSG SIZE rcvd: 123
root@ip-172-31-7-84:~# dig -x 193.0.14.129
; <<>> DiG 9.9.5-3-Ubuntu <<>> -x 193.0.14.129
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19459
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;129.14.0.193.in-addr.arpa. IN PTR
;; ANSWER SECTION:
129.14.0.193.in-addr.arpa. 21600 IN PTR k.root-servers.net.
;; Query time: 21 msec
;; SERVER: 172.31.0.2#53(172.31.0.2)
;; WHEN: Wed Jun 25 15:34:54 EST 2014
;; MSG SIZE rcvd: 86
root@ip-172-31-7-84:~# dig +tcp www.myrta.com
; <<>> DiG 9.9.5-3-Ubuntu <<>> +tcp www.myrta.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27172
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.myrta.com. IN A
;; ANSWER SECTION:
www.myrta.com. 30 IN A 163.189.7.175
;; Query time: 43 msec
;; SERVER: 172.31.0.2#53(172.31.0.2)
;; WHEN: Wed Jun 25 15:38:26 EST 2014
;; MSG SIZE rcvd: 58
root@ip-172-31-7-84:~# dig +trace www.myetoll.com
; <<>> DiG 9.9.5-3-Ubuntu <<>> +trace www.myetoll.com
;; global options: +cmd
. 18934 IN NS l.root-servers.net.
. 18934 IN NS m.root-servers.net.
. 18934 IN NS a.root-servers.net.
. 18934 IN NS b.root-servers.net.
. 18934 IN NS c.root-servers.net.
. 18934 IN NS d.root-servers.net.
. 18934 IN NS e.root-servers.net.
. 18934 IN NS f.root-servers.net.
. 18934 IN NS g.root-servers.net.
. 18934 IN NS h.root-servers.net.
. 18934 IN NS i.root-servers.net.
. 18934 IN NS j.root-servers.net.
. 18934 IN NS k.root-servers.net.
;; Received 239 bytes from 172.31.0.2#53(172.31.0.2) in 14 ms
com. 172800 IN NS k.gtld-servers.net.
com. 172800 IN NS c.gtld-servers.net.
com. 172800 IN NS j.gtld-servers.net.
com. 172800 IN NS e.gtld-servers.net.
com. 172800 IN NS a.gtld-servers.net.
com. 172800 IN NS l.gtld-servers.net.
com. 172800 IN NS m.gtld-servers.net.
com. 172800 IN NS f.gtld-servers.net.
com. 172800 IN NS b.gtld-servers.net.
com. 172800 IN NS i.gtld-servers.net.
com. 172800 IN NS h.gtld-servers.net.
com. 172800 IN NS g.gtld-servers.net.
com. 172800 IN NS d.gtld-servers.net.
com. 86400 IN DS 30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com. 86400 IN RRSIG DS 8 1 86400 20140701000000 20140623230000 40926 . lDy4cb2Mb6tyMYXfCNX1bcEwE2Rg6OkcsuSv+R81yFHEG5/luQvypHHr nt6apJYVf30t748Dtu5X3H56IXXlgftqgY93AbuAjugAz3gf8YiQtRW5 wQcKjtzq2luW1YIVYtujntwfC4om+sVxtCJbKT88cqQUwyTvKr8D6SxS QtA=
;; Received 739 bytes from 192.36.148.17#53(i.root-servers.net) in 237 ms
myetoll.com. 172800 IN NS dns1.rta.nsw.gov.au.
myetoll.com. 172800 IN NS dns2.rta.nsw.gov.au.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0QFMDQRCSRU0651QLVA1JQB21IF7UR NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20140630044832 20140623033832 56657 com. Co2ZGs6Fo2h2zlCv4wQS4E28nhjNxNQ/InfoiMq52MUH4/IJ9rh1Ot4C vuEmO3J1U4VVYUedu1EyFtBe6hGpajEfsntCYFA2EFbi2PgzAHAw8Ljr NCC24OnMBfcpbM6ew5UFd/157h6Tv/a+ilystE7+goWZi7q6Y6VNPWle B60=
RFJREQ8DI8OGS8V03LLBHDLV4J3EIPHK.com. 86400 IN NSEC3 1 1 0 - RFJVBCOQJUN2E7A6HEMG5MM79DAEKVMV NS DS RRSIG
RFJREQ8DI8OGS8V03LLBHDLV4J3EIPHK.com. 86400 IN RRSIG NSEC3 8 2 86400 20140701043512 20140624032512 56657 com. McX9ZVW1HVTK4ZkKSOOAj9pkHb8VzqKIbRNdcMWN8OJ75GaL/9II1bNx pXodneZpd+qtRe3TVhweJlv30zXEritWsN507cQDDsDoIgu+aNpgCzO7 louvilFNU88pOPPx4Wk1oTS9BNpNV/rcw0GWF/Yb/Y2Q/XTiyNl5NLe0 LiM=
;; Received 581 bytes from 192.12.94.30#53(e.gtld-servers.net) in 376 ms
www.myetoll.com. 7200 IN NS dns22.rta.nsw.gov.au.
www.myetoll.com. 7200 IN NS dns11.rta.nsw.gov.au.
;; Received 98 bytes from 163.189.23.22#53(dns2.rta.nsw.gov.au) in 26 ms
;; Received 12 bytes from 163.189.217.1#53(dns22.rta.nsw.gov.au) in 9 ms
root@ip-172-31-7-84:~# dig +short www.rms.nsw.gov.au
163.189.7.150
Subscribe to:
Posts (Atom)