Archive for September, 2009


1. It Doesn’t Crash
Linux has been time-proven to be a reliable operating system. Although the desktop is not a new place for Linux, most Linux-based systems have been used as servers and embedded systems. High-visibility Web sites such as Google use Linux-based systems, but you also can find Linux inside the TiVo set-top box in many livingrooms.
Linux has proved to be so reliable and secure that it is commonly found in dedicated firewall and router systems used by high-profile companies to secure their networks. For more than ten years, it has not been uncommon for Linux systems to run for months or years without needing a single reboot.

2. Viruses Are Few and Far Between
Although it is possible to create a virus to target Linux systems, the design of the system itself makes it very difficult to become infected. A single user could cause local damage to his or her files by running a virus on his or her system; however, this would be an isolated instance rather than something could spread out of control.
In addition, virtually all Linux vendors offer free on-line security updates. The general philosophy of the Linux community has been to address possible security issues before they become a problem rather than hoping the susceptibility will go unnoticed.

3. Virtually Hardware-Independent
Linux was designed and written to be easily portable to different hardware. For the desktop user, this means that Linux has been and likely always will be the first operating system to take advantage of advances in hardware technology such as AMD’s 64-bit processor chips.

4. Freedom of Choice
Linux offers freedom of choice as far as which manufacturer you purchase the software from as well as which application programs you wish to use. Being able to pick the manufacturer means you have a real choice as far as type of support you receive. Being open-source software, new manufacturers can enter the market to address customer needs.
Choice of application programs means that you can select the tools that best address your needs. For example, three popular word processors are available. All three are free and interoperate with Microsoft Word, but each offers unique advantages and disadvantages. The same is true of Web browsers.

5. Standards
Linux itself and many common applications follow open standards. This means an update on one system will not make other systems obsolete.

6. Applications, Applications, Applications
Each Linux distribution comes with hundreds and possibly thousands of application programs included. This alone can save you thousands of dollars for each desktop system you configure. Although this is a very small subset, consider that the OpenOffice.org office suite is included as well as the GIMP, a program similar to (and many people say more capable than Adobe Photoshop); Scribus, a document layout program similar to Quark Xpress; Evolution, an e-mail system equivalent to Microsoft’s Outlook Express; and hundreds more.
For the more technically inclined, development tools, such as compilers for the C, C++, Ada, Fortran, Pascal and other languages, are included as well as Perl, PHP and Python interpreters. Editors and versioning tools also are included in this category.
Whether you are looking for Instant Messaging clients, backup tools or Web site development packages, they likely are all included within your base Linux distribution.

7. Interoperability
More and more computers are being connected to networks. No system would be complete if it did not include tools to allow it to interoperate with computers running other operating systems. Once again, Linux is very strong in this area.
Linux includes Samba, software that allows Linux to act as a client on a Microsoft Windows-based network. In fact, Samba includes server facilities such that you could run a Linux system as the server for a group of Linux and Windows-based client systems.
In addition, Linux includes software to network with Apple networks and Novell’s Netware. NFS, the networking technology developed on UNIX systems also is included.

8. It’s a Community Relationship, Not a Customer Relationship
Other operating systems are the products of single vendors. Linux, on the other hand, is openly developed, and this technology is shared among vendors. This means you become part of a community rather than a customer of a single manufacturer. Also, the supplier community easily can adjust to the needs of various user communities rather than spouting a “one size fits all” philosophy.
This means you can select a Linux vendor that appears to best address your needs and feel confident that you could switch vendors at a later time without losing your investment–both in terms of costs and learning.

9. It’s Not How Big Your Processor Is…
Because of a combination of the internal design of Linux and development contributions from a diverse community, Linux tends to be more frugal in the use of computer resources. This may manifest itself in a single desktop system running faster with Linux than with another operating system, but the advantages go far beyond that. It is possible, for example, to configure a single Linux system to act as a terminal server and then use outdated hardware as what are called thin clients.
This server/thin client configuration makes it possible for older, less powerful hardware to share the resources of a single powerful system thus extending the life of older machines.

10. Linux Is Configurable
Linux is a true multi-user operating system. Each user can have his or her own individual configuration all on one computer. This includes the look of the desktop, what icons are displayed, what programs are started automatically when the user logs in and even what language the desktop is in.

Advertisements

Online Networking: Ecademy Blogs: Search

Shared via AddThis

LatencyTOP is a Linux* tool for software developers (both kernel and userspace), aimed at identifying where in the system latency is happening, and what kind of operation/action is causing the latency to happen so that the code can be changed to avoid the worst latency hiccups.

There are many types and causes of latency. LatencyTOP focuses on the type of latency that causes skips in audio, stutters in your desktop experience or that overloads your server (while you have plenty of CPU power left).

LatencyTOP focuses on the cases where the applications want to run and execute useful code, but there’s some resource that’s not currently available (and the kernel then blocks the process). This is done both on a system level and on a per process level, so that you can see what’s happening to the system, and which process is suffering and/or causing the delays

Note:- LatencyTOP needs a kernel that was built with the following two options and by defauly ubuntu intrepid kernel comes with this

CONFIG_HAVE_LATENCYTOP_SUPPORT=y

CONFIG_LATENCYTOP=y

Install LatencyTOP in Ubuntu

sudo apt-get install latencytop

This will complete the installation.

Using LatencyTOP

If you want to run latencytop use the following command from your terminal

sudo latencytop

Once it opens you should see similar to the following screen

One more screenshot

If you want more information about latencytop use the folowing command from your terminal

man latencytop

How to install a Line6 Guitar Port (and the toneport With a very big thanks to Myhrman from Source Forge for helping me get this thing working and a big thanks to Grabner for providing the drivers for something that seems to be non existant for linux! This is to get the Guitar Port to work as a basic USB sound device so you will get a dry signal in and out that will work under OSS and Jack.

Note: Myhrman says it should work with most toneports too but neither of us have one to test it out.

First open your Terminal

Applications> Accessories> Terminal

You will need to get Subversion

sudo apt-get install subversion

Then you will need to get the source

svn co https://line6linux.svn.sourceforge.net/svnroot/line6linux

Change to the directory

cd line6linux/driver/trunk

Time to build from the source but first make sure you have the latest build and headers

sudo apt-get install build-essential
sudo apt-get install linux-headers

(Note: You will atlest have version 2.6.27.14. To check your version in terminal type uname -r)

Now that is updated and you are in the trunk directory

make
sudo make install

Now shutdown and restart with the guitar port (or toneport) connected and you should beable to see it in your
System> Preferences> Sound

Only use the OSS drivers for native OS support and to prevent feedback while using Jack Keep the “Monitor” channel muted.

It’s a given that all disks eventually die, and it’s easy to see why. The platters in a modern disk drive rotate more than a hundred times per second, maintaining submicron tolerances between the disk heads and the magnetic media that store data. Often they run 24/7 in dusty, overheated environments, thrashing on heavily loaded or poorly managed machines. So, it’s not surprising that experienced users are all too familiar with the symptoms of a dying disk. Strange things start happening. Inscrutable kernel error messages cover the console and then the system becomes unstable and locks up. Often, entire days are lost repeating recent work, re-installing the OS and trying to recover data. Even if you have a recent backup, sudden disk failure is a minor catastrophe.

Many users and system administrators don’t know that Self-Monitoring, Analysis and Reporting Technology systems (SMART) are built in to most modern ATA and SCSI hard disks. SMART disk drives internally monitor their own health and performance. In many cases, the disk itself provides advance warning that something is wrong, helping to avoid the scenario described above. Most implementations of SMART also allow users to perform self-tests on the disk and to monitor a number of performance and reliability attributes.

By profession I am a physicist. My research group runs a large computing cluster with 300 nodes and 600 disk drives, on which more than 50TB of physics data are stored. I became interested in SMART several years ago when I realized it could help reduce downtime and keep our cluster operating more reliably. For about a year I have been maintaining an open-source package called smartmontools, a spin-off of the UCSC smartsuite package, for this purpose.

In this article, I explain how to use smartmontools’ smartctl utility and smartd dæmon to monitor the health of a system’s disks. See smartmontools.sourceforge.net for download and installation instructions and consult the WARNINGS file for a list of problem disks/controllers. Additional documentation can be found in the man pages (man smartctl and man smartd) and on the Web page.

Versions of smartmontools are available for Slackware, Debian, SuSE, Mandrake, Gentoo, Conectiva and other Linux distributions. Red Hat’s existing products contain the UCSC smartsuite versions of smartctl and smartd, but the smartmontools versions will be included in upcoming releases.

To understand how smartmontools works, it’s helpful to know the history of SMART. The original SMART spec (SFF-8035i) was written by a group of disk drive manufacturers. In Revision 2 (April 1996) disks keep an internal list of up to 30 Attributes corresponding to different measures of performance and reliability, such as read and seek error rates. Each Attribute has a one-byte normalized value ranging from 1 to 253 and a corresponding one-byte threshold. If one or more of the normalized Attribute values less than or equal to its corresponding threshold, then either the disk is expected to fail in less than 24 hours or it has exceeded its design or usage lifetime. Some of the Attribute values are updated as the disk operates. Others are updated only through off-line tests that temporarily slow down disk reads/writes and, thus, must be run with a special command. In late 1995, parts of SFF-8035i were merged into the ATA-3 standard.

Starting with the ATA-4 standard, the requirement that disks maintain an internal Attribute table was dropped. Instead, the disks simply return an OK or NOT OK response to an inquiry about their health. A negative response indicates the disk firmware has determined that the disk is likely to fail. The ATA-5 standard added an ATA error log and commands to run disk self-tests to the SMART command set.

To make use of these disk features, you need to know how to use smartmontools to examine the disk’s Attributes (most disks are backward-compatible with SFF-8035i), query the disk’s health status, run disk self-tests, examine the disk’s self-test log (results of the last 21 self-tests) and examine the disk’s ATA error log (details of the last five disk errors). Although this article focuses on ATA disks, additional documentation about SCSI devices can be found on the smartmontools Web page.

To begin, give the command smartctl -a /dev/hda, using the correct path to your disk, as root. If SMART is not enabled on the disk, you first must enable it with the -s on option. You then see output similar to the output shown in Listings 1–5.

The first part of the output (Listing 1) lists model/firmware information about the disk—this one is an IBM/Hitachi GXP-180 example. Smartmontools has a database of disk types. If your disk is in the database, it may be able to interpret the raw Attribute values correctly.

Listing 1. Output of smartctl -i /dev/hda

Device Model:     IC35L120AVV207-0
Serial Number:    VNVD02G4G3R72G
Firmware Version: V24OA63A
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   6
ATA Standard is:  ATA/ATAPI-6 T13 1410D revision 3a
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

The second part of the output (Listing 2) shows the results of the health status inquiry. This is the one-line Executive Summary Report of disk health; the disk shown here has passed. If your disk health status is FAILING, back up your data immediately. The remainder of this section of the output provides information about the disk’s capabilities and the estimated time to perform short and long disk self-tests.

Listing 2. Output of smartctl -Hc /dev/hda

SMART overall-health self-assessment test result: PASSED

General SMART Values:
Off-line data collection status: (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Off-line Data Collection:
Enabled.
Self-test execution status:      (   0) The previous self-test routine
                                        completed without error or no
                                        self-test has ever been run.
Total time to complete off-line
data collection:                 (2855) seconds.
Offline data collection
capabilities:                    (0x1b) SMART execute Offline immediate.
                                        Automatic timer ON/OFF support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        No Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (  48) minutes.

The third part of the output (Listing 3) lists the disk’s table of up to 30 Attributes (from a maximum set of 255). Remember that Attributes are no longer part of the ATA standard, but most manufacturers still support them. Although SFF-8035i doesn’t define the meaning or interpretation of Attributes, many have a de facto standard interpretation. For example, this disk’s 13th Attribute (ID #194) tracks its internal temperature.

Listing 3. Output of smartctl -A /dev/hda

Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate       0x000b   100   100   060    Pre-fail  Always      -       0
2 Throughput_Performance    0x0005   155   155   050    Pre-fail  Offline     -       225
3 Spin_Up_Time              0x0007   097   097   024    Pre-fail  Always      -       293 (Average 270)
4 Start_Stop_Count          0x0012   100   100   000    Old_age   Always      -       10
5 Reallocated_Sector_Ct     0x0033   100   100   005    Pre-fail  Always      -       0
7 Seek_Error_Rate           0x000b   100   100   067    Pre-fail  Always      -       0
8 Seek_Time_Performance     0x0005   125   125   020    Pre-fail  Offline     -       36
9 Power_On_Hours            0x0012   100   100   000    Old_age   Always      -       3548
10 Spin_Retry_Count         0x0013   100   100   060    Pre-fail  Always      -       0
12 Power_Cycle_Count        0x0032   100   100   000    Old_age   Always      -       10
192 Power-Off_Retract_Count 0x0032   100   100   050    Old_age   Always      -       158
193 Load_Cycle_Count        0x0012   100   100   050    Old_age   Always      -       158
194 Temperature_Celsius     0x0002   189   189   000    Old_age   Always      -       29 (Lifetime Min/Max 23/33)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always      -       0
197 Current_Pending_Sector  0x0022   100   100   000    Old_age   Always      -       0
198 Offline_Uncorrectable   0x0008   100   100   000    Old_age   Offline     -       0
199 UDMA_CRC_Error_Count    0x000a   200   200   000    Old_age   Always      -       0

Studies have shown that lowering disk temperatures by as little as 5°C significantly reduces failure rates, though this is less of an issue for the latest generation of fluid-drive bearing drives. One of the simplest and least expensive steps you can take to ensure disk reliability is to add a cooling fan that blows cooling air directly onto or past the system’s disks.

Each Attribute has a six-byte raw value (RAW_VALUE) and a one-byte normalized value (VALUE). In this case, the raw value stores three temperatures: the disk’s temperature in Celsius (29), plus its lifetime minimum (23) and maximum (33) values. The format of the raw data is vendor-specific and not specified by any standard. To track disk reliability, the disk’s firmware converts the raw value to a normalized value ranging from 1 to 253. If this normalized value is less than or equal to the threshold (THRESH), the Attribute is said to have failed, as indicated in the WHEN_FAILED column. The column is empty because none of these Attributes has failed. The lowest (WORST) normalized value also is shown; it is the smallest value attained since SMART was enabled on the disk. The TYPE of the Attribute indicates if Attribute failure means the device has reached the end of its design life (Old_age) or it’s an impending disk failure (Pre-fail). For example, disk spin-up time (ID #3) is a prefailure Attribute. If this (or any other prefail Attribute) fails, disk failure is predicted in less than 24 hours.

The names/meanings of Attributes and the interpretation of their raw values is not specified by any standard. Different manufacturers sometimes use the same Attribute ID for different purposes. For this reason, the interpretation of specific Attributes can be modified using the -v option to smartctl; please see the man page for details. For example, some disks use Attribute 9 to store the power-on time of the disk in minutes; the -v 9,minutes option to smartctl correctly modifies the Attribute’s interpretation. If your disk model is in the smartmontools database, these -v options are set automatically.

The next part of the smartctl -a output (Listing 4) is a log of the disk errors. This particular disk has been error-free, and the log is empty. Typically, one should worry only if disk errors start to appear in large numbers. An occasional transient error that does not recur usually is benign. The smartmontools Web page has a number of examples of smartctl -a output showing some illustrative error log entries. They are timestamped with the disk’s power-on lifetime in hours when the error occurred, and the individual ATA commands leading up to the error are timestamped with the time in milliseconds after the disk was powered on. This shows whether the errors are recent or old.

Listing 4. Output of smartctl -l error /dev/hda

SMART Error Log Version: 1
No Errors Logged

The final part of the smartctl output (Listing 5) is a report of the self-tests run on the disk. These show two types of self-tests, short and long. (ATA-6/7 disks also may have conveyance and selective self-tests.) These can be run with the commands smartctl -t short /dev/hda and smartctl -t long /dev/hda and do not corrupt data on the disk. Typically, short tests take only a minute or two to complete, and long tests take about an hour. These self-tests do not interfere with the normal functioning of the disk, so the commands may be used for mounted disks on a running system. On our computing cluster nodes, a long self-test is run with a cron job early every Sunday morning. The entries in Listing 5 all are self-tests that completed without errors; the LifeTime column shows the power-on age of the disk when the self-test was run. If a self-test finds an error, the Logical Block Address (LBA) shows where the error occurred on the disk. The Remaining column shows the percentage of the self-test remaining when the error was found. If you suspect that something is wrong with a disk, I strongly recommend running a long self-test to look for problems.

Listing 5. Output of smartctl -l selftest /dev/hda

SMART Self-test log, version number 1
Num  Test_Description    Status       Remaining  LifeTime(hours)
LBA_of_first_error
# 1  Extended off-line   Completed          00%      3525         -
# 2  Extended off-line   Completed          00%      3357         -
# 3  Short off-line      Completed          00%      3059         -

The smartctl -t offline command can be used to carry out off-line tests. These off-line tests do not make entries in the self-test log. They date back to the SFF-8035i standard, and update values of the Attributes that are not updated automatically under normal disk operation (see the UPDATED column in Listing 3). Some disks support automatic off-line testing, enabled by smartctl -o on, which automatically runs an off-line test every few hours.

The SMART standard provides a mechanism for running disk self-tests and for monitoring aspects of disk performance. Its main shortcoming is that it doesn’t provide a direct mechanism for informing the OS or user if problems are found. In fact, because disk SMART status frequently is not monitored, many disk problems go undetected until they lead to catastrophic failure. Of course, you can monitor disks on a regular basis using the smartctl utility, as I’ve described, but this is a nuisance.

The remaining part of the smartmontools package is the smartd dæmon that does regular monitoring for you. It monitors the disk’s SMART data for signs of problems. It can be configured to send e-mail to users or system administrators or to run arbitrary scripts if problems are detected. By default, when smartd is started, it registers the system’s disks. It then checks their status every 30 minutes for failing Attributes, failing health status or increased numbers of ATA errors or failed self-tests and logs this information with SYSLOG in /var/log/messages by default.

You can control and fine-tune the behavior of smartd using the configuration file /etc/smartd.conf. This file is read when smartd starts up, before it forks into the background. Each line contains Directives pertaining to a different disk. The configuration file on our computing cluster nodes look like this:

# /etc/smartd.conf config file
/dev/hda -S on -o on -a -I 194 -m sense@phys.uwm.edu
/dev/hdc -S on -o on -a -I 194 -m sense@phys.uwm.edu

The first column indicates the device to be monitored. The -o on Directive enables the automatic off-line testing, and the -S on Directive enables automatic Attribute autosave. The -m Directive is followed by an e-mail address to which warning messages are sent, and the -a Directive instructs smartd to monitor all SMART features of the disk. In this configuration, smartd logs changes in all normalized attribute values. The -I 194 Directive means ignore changes in Attribute #194, because disk temperatures change often, and it’s annoying to have such changes logged on a regular basis.

Normally smartd is started by the normal UNIX init mechanism. For example, on Red Hat distributions, /etc/rc.d/init.d/smartd start and /etc/rc.d/init.d/smartd stop can be used to start and stop the dæmon.

Further information about the smartd and its config file can be found in the man page (man smartd), and summaries can be found with the commands smartd -D and smartd -h. For example, the -M test Directive sends a test e-mail warning message to confirm that warning e-mail messages are delivered correctly. Other Directives provide additional flexibility, such as monitoring changes in raw Attribute values.

What should you do if a disk shows signs of problems? What if a disk self-test fails or the disk’s SMART health status fails? Start by getting your data off the disk and on to another system as soon as possible. Second, run some extended disk self-tests and see if the problem is repeatable at the same LBA. If so, something probably is wrong with the disk. If the disk has failing SMART health status and is under warranty, the vendor usually will replace it. If the disk is failing its self-tests, many manufacturers provide specialized disk health programs, for example, Maxtor’s PowerMax or IBM’s Drive Fitness Test. Sometimes these programs actually can repair a disk by remapping bad sectors. Often, they report a special error code that can be used to get a replacement disk.

This article has covered the basics of smartmontools. To learn more, read the man pages and Web page, and then write to the support mailing list if you need further help. Remember, smartmontools is no substitute for backing up your data. SMART cannot and does not predict all disk failures, but it often provides clues that something is amiss and has helped to keep many computing clusters operating reliably.

Several developers are porting smartmontools to FreeBSD, Darwin and Solaris, and we recently have added extensions to allow smartmontools to monitor and control the ATA disks behind 3ware RAID controllers. If you would like to contribute to the development of smartmontools, write to the support mailing list. We especially are interested in information about the interpretation and meaning of vendor-specific SMART Attribute and raw values.

I often need to use bash, sometimes I don’t remember using of program, here are some often helpful scripts:

  • netstat -an | grep ESTABLISHED – shows all established connections
  • ps -aux | grep nobody – displays all processes of user nobody
  • who (or better: w) – who is logged on the machine
  • nmap host – shows open ports
  • find . -name ‘*.txt’ | xargs grep java – recursively finds all ‘java’ occurrences in files with txt extension
  • grep -R java . – recursively finds all ’java’ occurrences in all type files
  • nmap -sP 192.168.1.1-254 – shows all IP’s, that are up
  • nmblookup -A 192.168.1.1 – displays name of PC in the local network
  • scp -r root@host_name:/path /myPath – remotely copying files (-r – recurcively)
  • host 192.168.1.1 – displays host name
  • ping domain – displays IP number
  • smbclient -L hostname – shows Windows resources for Linux
  • wc -l file – shows number of lines in the file

Q. How do I add cron job under Linux or UNIX like operating system?

A. Cron job are used to schedule commands to be executed periodically i.e. to setup commands which will repeatedly run at a set time, you can use the cron jobs.

crontab is the command used to install, deinstall or list the tables used to drive the cron daemon in Vixie Cron. Each user can have their own crontab, and though these are files in /var/spool/cron/crontabs, they are not intended to be edited directly. You need to use crontab command for editing or setting up your own cron jobs.

To edit your crontab file, type the following command:
$ crontab -e

Syntax of crontab

Your cron job looks like as follows:
1 2 3 4 5 /path/to/command arg1 arg2

Where,

  • 1: Minute (0-59)
  • 2: Hours (0-23)
  • 3: Day (0-31)
  • 4: Month (0-12 [12 == December])
  • 5: Day of the week(0-7 [7 or 0 == sunday])
  • /path/to/command – Script or command name to schedule

Same above five fields structure can be easily remembered with following diagram:

* * * * * command to be executed
- - - - -
| | | | |
| | | | ----- Day of week (0 - 7) (Sunday=0 or 7)
| | | ------- Month (1 - 12)
| | --------- Day of month (1 - 31)
| ----------- Hour (0 - 23)
------------- Minute (0 - 59)

Example(s)
If you wished to have a script named /root/backup.sh run every day at 3am, my crontab entry would look like as follows:
(a) Install your cronjob:# crontab -e(b)Append following entry:0 3 * * * /root/backup.shRun five minutes after midnight, every day:5 0 * * * /path/to/commandRun at 2:15pm on the first of every month:15 14 1 * * /path/to/commandRun at 10 pm on weekdays: 0 22 * * 1-5 /path/to/command Run 23 minutes after midnigbt, 2am, 4am …, everyday:23 0-23/2 * * * /path/to/commandRun at 5 after 4 every sunday:5 4 * * sun /path/to/command

Use of operators

An operator allows you to specifying multiple values in a field. There are three operators:

  1. The asterisk (*) : This operator specifies all possible values for a field. For example, an asterisk in the hour time field would be equivalent to every hour or an asterisk in the month field would be equivalent to every month.
  2. The comma (,) : This operator specifies a list of values, for example: “1,5,10,15,20, 25”.
  3. The dash (-) : This operator specifies a range of values, for example: “5-15” days , which is equivalent to typing “5,6,7,8,9,….,13,14,15” using the comma operator.

How do I disabling Email output?

By default the output of a command or a script (if any produced), will be email to your local email account. To stop receiving email output from crontab you need to append >/dev/null 2>&1. For example:0 3 * * * /root/backup.sh >/dev/null 2>&1To mail output to particluer email account let us say vivek@nixcraft.in you need to define MAILTO variable to your cron job:MAILTO="vivek@nixcraft.in"
0 3 * * * /root/backup.sh >/dev/null 2>&1

Task:To list your crontab jobs use the command

Type the following command:# crontab -lTo remove or erase all crontab jobs use the command:# crontab -r

Use special string to save time

Instead of the first five fields, you can use any one of eight special strings. It will not just save your time but it will improve readability.

Special string Meaning
@reboot Run once, at startup.
@yearly Run once a year, “0 0 1 1 *”.
@annually (same as @yearly)
@monthly Run once a month, “0 0 1 * *”.
@weekly Run once a week, “0 0 * * 0”.
@daily Run once a day, “0 0 * * *”.
@midnight (same as @daily)
@hourly Run once an hour, “0 * * * *”.

Run ntpdate every hour:
@hourly /path/to/ntpdate
Make a backup everyday:
@daily /path/to/backup/script.sh

Understanding /etc/crontab file and /etc/cron.d/* directories

/etc/crontab is system crontabs file. Usually only used by root user or daemons to configure system wide jobs. All individual user must must use crontab command to install and edit their jobs as described above. /var/spool/cron/ or /var/cron/tabs/ is directory for personal user crontab files. It must be backup with users home directory.

Typical /etc/crontab file entries:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly

Additionally, cron reads the files in /etc/cron.d/ directory. Usually system daemon such as sa-update or sysstat places their cronjob here. As a root user or superuser you can use following directories to configure cronjobs. You can directly drop your scripts here. run-parts command run scripts or programs in a directory via /etc/crontab

Directory Description
/etc/cron.d/ Put all scripts here and call them from /etc/crontab file.
/etc/cron.daily/ Run all scripts once a day
/etc/cron.hourly/ Run all scripts once an hour
/etc/cron.monthly/ Run all scripts once a month
/etc/cron.weekly/ Run all scripts once a week

How do I use above directories to put scripts?

Here is a sample shell script (clean.cache) to clean up cached files every 10 days. This script is directly created at /etc/cron.daliy/ directory i.e. create a file called /etc/cron.daily/clean.cache:

 #!/bin/bash
CROOT="/tmp/cachelighttpd/"
DAYS=10
LUSER="lighttpd"
LGROUP="lighttpd"

# start cleaning
/usr/bin/find ${CROOT} -type f -mtime +${DAYS} | xargs -r /bin/rm

# if directory deleted by some other script just get it back
if [ ! -d $CROOT ]
then
        /bin/mkdir -p $CROOT
        /bin/chown ${LUSER}:${LGROUP} ${CROOT}
fi

Need to monitor Linux server performance? Try these built-in command and a few add-on tools. Most Linux distributions are equipped with tons of monitoring. These tools provide metrics which can be used to get information about system activities. You can use these tools to find the possible causes of a performance problem. The commands discussed below are some of the most basic commands when it comes to system analysis and debugging server issues such as:

  1. Finding out bottlenecks.
  2. Disk (storage) bottlenecks.
  3. CPU and memory bottlenecks.
  4. Network bottlenecks.


#1: top – Process Activity Command

The top program provides a dynamic real-time view of a running system i.e. actual process activity. By default, it displays the most CPU-intensive tasks running on the server and updates the list every five seconds.

Fig.01: Linux top commandFig.01: Linux top command

Commonly Used Hot Keys

The top command provides several useful hot keys:

Hot Key Usage
t Displays summary information off and on.
m Displays memory information off and on.
A Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system.
f Enters an interactive configuration screen for top. Helpful for setting up top for a specific task.
o Enables you to interactively select the ordering within top.
r Issues renice command.
k Issues kill command.
z Turn on or off color/mono

=> Related: How do I Find Out Linux CPU Utilization?

#2: vmstat – System Activity, Hardware and System Information

The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
# vmstat 3
Sample Outputs:

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 2540988 522188 5130400    0    0     2    32    4    2  4  1 96  0  0
 1  0      0 2540988 522188 5130400    0    0     0   720 1199  665  1  0 99  0  0
 0  0      0 2540956 522188 5130400    0    0     0     0 1151 1569  4  1 95  0  0
 0  0      0 2540956 522188 5130500    0    0     0     6 1117  439  1  0 99  0  0
 0  0      0 2540940 522188 5130512    0    0     0   536 1189  932  1  0 98  0  0
 0  0      0 2538444 522188 5130588    0    0     0     0 1187 1417  4  1 96  0  0
 0  0      0 2490060 522188 5130640    0    0     0    18 1253 1123  5  1 94  0  0

Display Memory Utilization Slabinfo

# vmstat -m

Get Information About Active / Inactive Memory Pages

# vmstat -a
=> Related: How do I find out Linux Resource utilization to detect system bottlenecks?

#3: w – Find Out Who Is Logged on And What They Are Doing

w command displays information about the users currently on the machine, and their processes.
# w username
# w vivek

Sample Outputs:

 17:58:47 up 5 days, 20:28,  2 users,  load average: 0.36, 0.26, 0.24
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    10.1.3.145       14:55    5.00s  0.04s  0.02s vim /etc/resolv.conf
root     pts/1    10.1.3.145       17:43    0.00s  0.03s  0.00s w

#4: uptime – Tell How Long The System Has Been Running

The uptime command can be used to see how long the server has been running. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.
# uptime
Output:

 18:02:41 up 41 days, 23:42,  1 user,  load average: 0.00, 0.00, 0.00

1 can be considered as optimal load value. The load can change from system to system. For a single CPU system 1 – 3 and SMP systems 6-10 load value might be acceptable.

#5: ps – Displays The Processes

ps command will report a snapshot of the current processes. To select all processes use the -A or -e option:
# ps -A
Sample Outputs:

  PID TTY          TIME CMD
    1 ?        00:00:02 init
    2 ?        00:00:02 migration/0
    3 ?        00:00:01 ksoftirqd/0
    4 ?        00:00:00 watchdog/0
    5 ?        00:00:00 migration/1
    6 ?        00:00:15 ksoftirqd/1
....
.....
 4881 ?        00:53:28 java
 4885 tty1     00:00:00 mingetty
 4886 tty2     00:00:00 mingetty
 4887 tty3     00:00:00 mingetty
 4888 tty4     00:00:00 mingetty
 4891 tty5     00:00:00 mingetty
 4892 tty6     00:00:00 mingetty
 4893 ttyS1    00:00:00 agetty
12853 ?        00:00:00 cifsoplockd
12854 ?        00:00:00 cifsdnotifyd
14231 ?        00:10:34 lighttpd
14232 ?        00:00:00 php-cgi
54981 pts/0    00:00:00 vim
55465 ?        00:00:00 php-cgi
55546 ?        00:00:00 bind9-snmp-stat
55704 pts/1    00:00:00 ps

ps is just like top but provides more information.

Show Long Format Output

# ps -Al
To turn on extra full mode (it will show command line arguments passed to process):
# ps -AlF

To See Threads ( LWP and NLWP)

# ps -AlFH

To See Threads After Processes

# ps -AlLm

Print All Process On The Server

# ps ax
# ps axu

Print A Process Tree

# ps -ejH
# ps axjf
# pstree

Print Security Information

# ps -eo euser,ruser,suser,fuser,f,comm,label
# ps axZ
# ps -eM

See Every Process Running As User Vivek

# ps -U vivek -u vivek u

Set Output In a User-Defined Format

# ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
# ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
# ps -eopid,tt,user,fname,tmout,f,wchan

Display Only The Process IDs of Lighttpd

# ps -C lighttpd -o pid=
OR
# pgrep lighttpd
OR
# pgrep -u vivek php-cgi

Display The Name of PID 55977

# ps -p 55977 -o comm=

Find Out The Top 10 Memory Consuming Process

# ps -auxf | sort -nr -k 4 | head -10

Find Out top 10 CPU Consuming Process

# ps -auxf | sort -nr -k 3 | head -10

#6: free – Memory Usage

The command free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
# free
Sample Output:

            total       used       free     shared    buffers     cached
Mem:      12302896    9739664    2563232          0     523124    5154740
-/+ buffers/cache:    4061800    8241096
Swap:      1052248          0    1052248

=> Related: :

  1. Linux Find Out Virtual Memory PAGESIZE
  2. Linux Limit CPU Usage Per Process
  3. How much RAM does my Ubuntu / Fedora Linux desktop PC have?

#7: iostat – Average CPU Load, Disk Activity

The command iostat report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).
# iostat
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 	06/26/2009

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.50    0.09    0.51    0.03    0.00   95.86

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              22.04        31.88       512.03   16193351  260102868
sda1              0.00         0.00         0.00       2166        180
sda2             22.04        31.87       512.03   16189010  260102688
sda3              0.00         0.00         0.00       1615          0

=> Related: : Linux Track NFS Directory / Disk I/O Stats

#8: sar – Collect and Report System Activity

The sar command is used to collect, report, and save system activity information. To see network counter, enter:
# sar -n DEV | more
To display the network counters from the 24th:
# sar -n DEV -f /var/log/sa/sa24 | more
You can also display real time usage using sar:
# sar 4 5
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 		06/26/2009

06:45:12 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle
06:45:16 PM       all      2.00      0.00      0.22      0.00      0.00     97.78
06:45:20 PM       all      2.07      0.00      0.38      0.03      0.00     97.52
06:45:24 PM       all      0.94      0.00      0.28      0.00      0.00     98.78
06:45:28 PM       all      1.56      0.00      0.22      0.00      0.00     98.22
06:45:32 PM       all      3.53      0.00      0.25      0.03      0.00     96.19
Average:          all      2.02      0.00      0.27      0.01      0.00     97.70

=> Related: : How to collect Linux system utilization data into a file

#9: mpstat – Multiprocessor Usage

The mpstat command displays activities for each available processor, processor 0 being the first one. mpstat -P ALL to display average CPU utilization per processor:
# mpstat -P ALL
Sample Output:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in)	 	06/26/2009

06:48:11 PM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
06:48:11 PM  all    3.50    0.09    0.34    0.03    0.01    0.17    0.00   95.86   1218.04
06:48:11 PM    0    3.44    0.08    0.31    0.02    0.00    0.12    0.00   96.04   1000.31
06:48:11 PM    1    3.10    0.08    0.32    0.09    0.02    0.11    0.00   96.28     34.93
06:48:11 PM    2    4.16    0.11    0.36    0.02    0.00    0.11    0.00   95.25      0.00
06:48:11 PM    3    3.77    0.11    0.38    0.03    0.01    0.24    0.00   95.46     44.80
06:48:11 PM    4    2.96    0.07    0.29    0.04    0.02    0.10    0.00   96.52     25.91
06:48:11 PM    5    3.26    0.08    0.28    0.03    0.01    0.10    0.00   96.23     14.98
06:48:11 PM    6    4.00    0.10    0.34    0.01    0.00    0.13    0.00   95.42      3.75
06:48:11 PM    7    3.30    0.11    0.39    0.03    0.01    0.46    0.00   95.69     76.89

=> Related: : Linux display each multiple SMP CPU processors utilization individually.

#10: pmap – Process Memory Usage

The command pmap report memory map of a process. Use this command to find out causes of memory bottlenecks.
# pmap -d PID
To display process memory information for pid # 47394, enter:
# pmap -d 47394
Sample Outputs:

47394:   /usr/bin/php-cgi
Address           Kbytes Mode  Offset           Device    Mapping
0000000000400000    2584 r-x-- 0000000000000000 008:00002 php-cgi
0000000000886000     140 rw--- 0000000000286000 008:00002 php-cgi
00000000008a9000      52 rw--- 00000000008a9000 000:00000   [ anon ]
0000000000aa8000      76 rw--- 00000000002a8000 008:00002 php-cgi
000000000f678000    1980 rw--- 000000000f678000 000:00000   [ anon ]
000000314a600000     112 r-x-- 0000000000000000 008:00002 ld-2.5.so
000000314a81b000       4 r---- 000000000001b000 008:00002 ld-2.5.so
000000314a81c000       4 rw--- 000000000001c000 008:00002 ld-2.5.so
000000314aa00000    1328 r-x-- 0000000000000000 008:00002 libc-2.5.so
000000314ab4c000    2048 ----- 000000000014c000 008:00002 libc-2.5.so
.....
......
..
00002af8d48fd000       4 rw--- 0000000000006000 008:00002 xsl.so
00002af8d490c000      40 r-x-- 0000000000000000 008:00002 libnss_files-2.5.so
00002af8d4916000    2044 ----- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b15000       4 r---- 0000000000009000 008:00002 libnss_files-2.5.so
00002af8d4b16000       4 rw--- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b17000  768000 rw-s- 0000000000000000 000:00009 zero (deleted)
00007fffc95fe000      84 rw--- 00007ffffffea000 000:00000   [ stack ]
ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
mapped: 933712K    writeable/private: 4304K    shared: 768000K

The last line is very important:

  • mapped: 933712K total amount of memory mapped to files
  • writeable/private: 4304K the amount of private address space
  • shared: 768000K the amount of address space this process is sharing with others

=> Related: : Linux find the memory used by a program / process using pmap command

#11 and #12: netstat and ss – Network Statistics

The command netstat displays network connections, routing tables, interface statistics, masquerade connections, and multicast memberships. ss command is used to dump socket statistics. It allows showing information similar to netstat. See the following resources about ss and netstat commands:

#13: iptraf – Real-time Network Statistics

The iptraf command is interactive colorful IP LAN monitor. It is an ncurses-based IP LAN monitor that generates various network statistics including TCP info, UDP counts, ICMP and OSPF information, Ethernet load info, node stats, IP checksum errors, and others. It can provide the following info in easy to read format:

  • Network traffic statistics by TCP connection
  • IP traffic statistics by network interface
  • Network traffic statistics by protocol
  • Network traffic statistics by TCP/UDP port and by packet size
  • Network traffic statistics by Layer2 address
Fig.02: General interface statistics: IP traffic statistics by network interface Fig.02: General interface statistics: IP traffic statistics by network interface
Fig.03 Network traffic statistics by TCP connectionFig.03 Network traffic statistics by TCP connection

#14: tcpdump – Detailed Network Traffic Analysis

The tcpdump is simple command that dump traffic on a network. However, you need good understanding of TCP/IP protocol to utilize this tool. For.e.g to display traffic info about DNS, enter:
# tcpdump -i eth1 'udp port 53'
To display all IPv4 HTTP packets to and from port 80, i.e. print only packets that contain data, not, for example, SYN and FIN packets and ACK-only packets, enter:
# tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
To display all FTP session to 202.54.1.5, enter:
# tcpdump -i eth1 'dst 202.54.1.5 and (port 21 or 20'
To display all HTTP session to 192.168.1.5:
# tcpdump -ni eth0 'dst 192.168.1.5 and tcp and port http'
Use wireshark to view detailed information about files, enter:
# tcpdump -n -i eth1 -s 0 -w output.txt src or dst port 80

#15: strace – System Calls

Trace system calls and signals. This is useful for debugging webserver and other server problems. See how to use to trace the process and see What it is doing.

#16: /Proc file system – Various Kernel Statistics

/proc file system provides detailed information about various hardware devices and other Linux kernel information. See Linux kernel /proc documentations for further details. Common /proc examples:
# cat /proc/cpuinfo
# cat /proc/meminfo
# cat /proc/zoneinfo
# cat /proc/mounts

17#: Nagios – Server And Network Monitoring

Nagios is a popular open source computer system and network monitoring application software. You can easily monitor all your hosts, network equipment and services. It can send alert when things go wrong and again when they get better. FAN is “Fully Automated Nagios”. FAN goals are to provide a Nagios installation including most tools provided by the Nagios Community. FAN provides a CDRom image in the standard ISO format, making it easy to easilly install a Nagios server. Added to this, a wide bunch of tools are including to the distribution, in order to improve the user experience around Nagios.

18#: Cacti – Web-based Monitoring Tool

Cacti is a complete network graphing solution designed to harness the power of RRDTool’s data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices. It can provide data about network, CPU, memory, logged in users, Apache, DNS servers and much more. See how to install and configure Cacti network graphing tool under CentOS / RHEL.

#19: KDE System Guard – Real-time Systems Reporting and Graphing

KSysguard is a network enabled task and system monitor application for KDE desktop. This tool can be run over ssh session. It provides lots of features such as a client/server architecture that enables monitoring of local and remote hosts. The graphical front end uses so-called sensors to retrieve the information it displays. A sensor can return simple values or more complex information like tables. For each type of information, one or more displays are provided. Displays are organized in worksheets that can be saved and loaded independently from each other. So, KSysguard is not only a simple task manager but also a very powerful tool to control large server farms.

Fig.05 KDE System GuardFig.05 KDE System Guard {Image credit: Wikipedia}

See the KSysguard handbook for detailed usage.

#20: Gnome System Monitor – Real-time Systems Reporting and Graphing

The System Monitor application enables you to display basic system information and monitor system processes, usage of system resources, and file systems. You can also use System Monitor to modify the behavior of your system. Although not as powerful as the KDE System Guard, it provides the basic information which may be useful for new users:

  • Displays various basic information about the computer’s hardware and software.
  • Linux Kernel version
  • GNOME version
  • Hardware
  • Installed memory
  • Processors and speeds
  • System Status
  • Currently available disk space
  • Processes
  • Memory and swap space
  • Network usage
  • File Systems
  • Lists all mounted filesystems along with basic information about each.
Fig.06 The Gnome System Monitor applicationFig.06 The Gnome System Monitor application

Bounce: Additional Tools

A few more tools:

  • nmap – scan your server for open ports.
  • lsof – list open files, network connections and much more.
  • ntop web based tool – ntop is the best tool to see network usage in a way similar to what top command does for processes i.e. it is network traffic monitoring software. You can see network status, protocol wise distribution of traffic for UDP, TCP, DNS, HTTP and other protocols.
  • Conky – Another good monitoring tool for the X Window System. It is highly configurable and is able to monitor many system variables including the status of the CPU, memory, swap space, disk storage, temperatures, processes, network interfaces, battery power, system messages, e-mail inboxes etc.
  • GKrellM – It can be used to monitor the status of CPUs, main memory, hard disks, network interfaces, local and remote mailboxes, and many other things.
  • vnstat – vnStat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interface(s).
  • htop – htop is an enhanced version of top, the interactive process viewer, which can display the list of processes in a tree form.
  • mtr – mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.