SAP Sybase ASE 15.7 Raw Device Setup Centos Linux Worked Example





SAP Sybase ASE 15.7 Raw Devices

This is a worked example of setting up raw devices on Centos Linux 6.4 for use with Sybase ASE 15.7.

Raw devices take more set up than filesystem devices and have been the usual method of setting up devices for production unix based systems for Sybase ASE.

For development systems setting up with filesystem devices using directio is a simpler approach.

For simplicity the example below shows the set up of only four raw devices for use as sybase devices.

SAP Support for Raw Devices

On the SAP discussion board there appears to be a difference in opinion as to whether raw devices are still supported.

Raw devices appear to be documented in the manual, which might imply support (See screenshot of manual).

SAP sybase manual ase raw devices 15.7

Arguments for Raw Devices

Sybase ASE writes directly to the raw device and this configuration is viewed as more resilient to system failure than directio device set up. Whilst the ASE product has moved on, the principle of trying to protect the master database from corruption due to a system failure still holds.

As a contractor I've worked on six different sites running Sybase and all of them used raw devices for the master database. Even the development systems were set up with raw devices.

Tests on the Sybase ASE 15.7 Centos system showed Raw devices to work significantly faster:

Sybase Database load: 4.4 times faster.
Sybase Insert Test 12.1 times faster.


Sybase ASE documentation appears to still endorse raw devices as preferred for Unix / Linux.

Using Raw Devices forces administrators to think more carefully about disk space planning for data and log.

Discussion on SAP Community Network on Sybase ASE Raw Devices

Here is a link to a discussion on the SAP Community Network:-

http://scn.sap.com/thread/3569074

It seems that most of the commentators on this thread have not actually done a test themselves. I very much doubt that a tweaked system could change the speed ratio of the insert test from 12.1 times faster to the same speed. In the real world due to separation of Unix teams and DBA teams the scope for tweaking and testing with changes to linux operating system settings is usually very limited.

Jeff Tallman gives a very detailed response at the bottom of the SAP Community Network page and is probably the only contributor worth taking notice of.

Unfortunately Jeff doesn't actually answer the question as to whether raw devices on linux ASE 15.7 are supported by SAP.

Jeff comments about being cautious about results from simplistic "toy" tests, which is a fair point.

SAP Document on Raw Devices

For SAP applications file system devices are preferred due to the set up required.

SAP pdf document on raw devices

Arguments against Raw Devices in favour of directio devices

Requires assistance and resources from a Unix System Administrator.

Many more steps involved which can take time and introduce error.

In recent years raw devices have been put on a list for deprecation in some linux systems and then reinstated.

Oracle has deprecated raw devices implying an industry move away from raw devices.

Centos linux raw devices require startup script "raw" commands to bind the devices. After a system reboot the devices will disappear from the /dev/raw directory and will require re-running the raw command after start up, permissions and ownership as detailed below.

1) Check Hard Disk Recognised After Hardware installation of the disk

In this example a second hard disk was installed.

After starting Centos again the disk was recognised and listed using fdisk -l as root.

The primary hard disk is /dev/sda, so the new second hard disk is automatically assigned at /dev/sdb.

[root@linuxdev ~]# fdisk -l

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

2) Use pvcreate to Create a Physical Volume

First use pvscan and pvdisplay to show existing physical volumes...

[root@linuxdev new_disk]# pvscan
PV /dev/sda2 VG vg_home lvm2 [111.27 GiB / 0 free]
Total: 1 [111.27 GiB] / in use: 1 [111.27 GiB] / in no VG: 0 [0 ]

[root@linuxdev dev]# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name vg_home
PV Size 111.27 GiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 28484
Free PE 0
Allocated PE 28484
PV UUID ony1C0-iWOs-xEJF-zap3-Cdt9-XbBO-lOPy25

[root@linuxdev dev]# pvcreate /dev/sdb
Physical volume "/dev/sdb" successfully created

[root@linuxdev dev]# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name vg_home
PV Size 111.27 GiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 28484
Free PE 0
Allocated PE 28484
PV UUID ony1C0-iWOs-xEJF-zap3-Cdt9-XbBO-lOPy25

"/dev/sdb" is a new physical volume of "465.76 GiB"
--- NEW Physical volume ---
PV Name /dev/sdb
VG Name
PV Size 465.76 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID 2NaomA-q7oK-TjtZ-OCpk-P5cI-qZfL-Df0Nbe

3) Use vgcreate to Create a Volume Group

Create a volume group with name "syb_vol_group1" on second hard disk /dev/sdb ...

[root@linuxdev dev]# vgcreate syb_vol_group1 /dev/sdb
Volume group "syb_vol_group1" successfully created

Show the volume group using vgdisplay...

[root@linuxdev dev]# vgdisplay
--- Volume group ---
VG Name syb_vol_group1
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 465.76 GiB
PE Size 4.00 MiB
Total PE 119234
Alloc PE / Size 0 / 0
Free PE / Size 119234 / 465.76 GiB
VG UUID z040Q6-JiEU-gzGA-nIbo-c9iq-9woJ-lIlWQP

Volume groups from the primary hard disk will also be displayed...

--- Volume group ---
VG Name vg_home
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 1
Act PV 1
VG Size 111.27 GiB
PE Size 4.00 MiB
Total PE 28484
Alloc PE / Size 28484 / 111.27 GiB
Free PE / Size 0 / 0
VG UUID 5LJewA-q6xC-53L3-j5yq-o3Nq-7q5n-2eHH5f

4) Use lvcreate to Create Logical Volumes

This will create a logical volume of 300Mb called syb_master_lv in the volume group syb_vol_group1 ...

[root@linuxdev dev]# lvcreate -L300 -n syb_master_lv syb_vol_group1
Logical volume "syb_master_lv" created

Display the new logical volume using lvdisplay...

[root@linuxdev dev]# lvdisplay
--- Logical volume ---
LV Path /dev/syb_vol_group1/syb_master_lv
LV Name syb_master_lv
VG Name syb_vol_group1
LV UUID SdC51H-L2Ge-pbIK-UkLV-n1ua-SNzT-cNtqLM
LV Write Access read/write
LV Creation host, time linuxdev.home, 2013-09-05 16:11:53 +0100
LV Status available
# open 0
LV Size 300.00 MiB
Current LE 75
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:3

Create other logical volumes...

lvcreate -L300 -n syb_sysprocs_lv syb_vol_group1
lvcreate -L500 -n syb_data001_lv syb_vol_group1
lvcreate -L500 -n syb_log001_lv syb_vol_group1

[root@linuxdev dev]# lvcreate -L300 -n syb_sysprocs_lv syb_vol_group1
Logical volume "syb_sysprocs_lv" created
[root@linuxdev dev]# lvcreate -L500 -n syb_data001_lv syb_vol_group1
Logical volume "syb_data001_lv" created
[root@linuxdev dev]# lvcreate -L500 -n syb_log001_lv syb_vol_group1
Logical volume "syb_log001_lv" created

5) Activate the Volume Group using vgchange

Activate volume group syb_vol_group1...

[root@linuxdev dev]# vgchange -a y syb_vol_group1
4 logical volume(s) in volume group "syb_vol_group1" now active

6) Create raw bindings using /bin/raw command

For raw devices each logical volume needs to be bound to a numbered raw device...

/bin/raw /dev/raw/raw1 /dev/syb_vol_group1/syb_master_lv
/bin/raw /dev/raw/raw2 /dev/syb_vol_group1/syb_sysprocs_lv
/bin/raw /dev/raw/raw3 /dev/syb_vol_group1/syb_data001_lv
/bin/raw /dev/raw/raw4 /dev/syb_vol_group1/syb_log001_lv

[root@linuxdev raw]# /bin/raw /dev/raw/raw1 /dev/syb_vol_group1/syb_master_lv
/dev/raw/raw1: bound to major 253, minor 3

[root@linuxdev raw]# /bin/raw /dev/raw/raw2 /dev/syb_vol_group1/syb_sysprocs_lv
/dev/raw/raw2: bound to major 253, minor 4

[root@linuxdev raw]# /bin/raw /dev/raw/raw3 /dev/syb_vol_group1/syb_data001_lv
/dev/raw/raw3: bound to major 253, minor 5

[root@linuxdev raw]# /bin/raw /dev/raw/raw4 /dev/syb_vol_group1/syb_log001_lv
/dev/raw/raw4: bound to major 253, minor 6

Change permissions on the raw devices...

/bin/chmod 600 /dev/raw/raw1
/bin/chmod 600 /dev/raw/raw2
/bin/chmod 600 /dev/raw/raw3
/bin/chmod 600 /dev/raw/raw4

Change ownership on the raw devices to sybase...

/bin/chown sybase:sybase /dev/raw/raw1
/bin/chown sybase:sybase /dev/raw/raw2
/bin/chown sybase:sybase /dev/raw/raw3
/bin/chown sybase:sybase /dev/raw/raw4

7) Make a directory for symbolic links to the devices for ease of administration

mkdir /home/sybase/devices
cd /home/sybase/devices

Create symbolic links to the raw devices
#ln -s source target

ln -s /dev/raw/raw1 master
ln -s /dev/raw/raw2 sysprocsdev
ln -s /dev/raw/raw3 data001
ln -s /dev/raw/raw4 log001

[root@linuxdev sybase]# ls -l /home/sybase/devices
total 0
lrwxrwxrwx 1 sybase sybase 13 Sep 5 16:56 data001 -> /dev/raw/raw3
lrwxrwxrwx 1 sybase sybase 13 Sep 5 16:57 log001 -> /dev/raw/raw4
lrwxrwxrwx 1 sybase sybase 13 Sep 5 16:54 master -> /dev/raw/raw1
lrwxrwxrwx 1 sybase sybase 13 Sep 5 16:56 sysprocsdev -> /dev/raw/raw2

8) Within sybase dataserver set up sybase devices with disk init

run disk init within sybase ASE...

disk init name="RAW_DATA001", size="500M",
physname="/home/sybase/devices/data001"
go

disk init name="RAW_LOG001", size="500M",
physname="/home/sybase/devices/log001"
go

create a test database to check ASE can allocate the pages...

create database raw_test on RAW_DATA001=50 log on RAW_LOG001=20
go
CREATE DATABASE: allocating 12800 logical pages (50.0 megabytes) on disk 'RAW_DATA001' (12800 logical pages requested).
CREATE DATABASE: allocating 5120 logical pages (20.0 megabytes) on disk 'RAW_LOG001' (5120 logical pages requested).
Database 'raw_test' is now online.

9) Set up Raw Device Binding in System Startup Script

Commands previously run in the set up above will need to be run on each system re-start.

Without re-run of these command Sybase ASE errorlog will show errors:-

00:0006:00000:00001:2013/09/06 10:45:00.03 server Activating disk 'RAW_DATA001' of size 512000 KB.
00:0006:00000:00001:2013/09/06 10:45:00.03 kernel dopen: open '/home/sybase/devices/data001', No such file or directory
00:0006:00000:00001:2013/09/06 10:45:00.03 kernel udactivate: error starting virtual disk -1
00:0006:00000:00001:2013/09/06 10:45:00.03 server Activating disk 'RAW_LOG001' of size 512000 KB.
00:0006:00000:00001:2013/09/06 10:45:00.03 kernel dopen: open '/home/sybase/devices/log001', No such file or directory
00:0006:00000:00001:2013/09/06 10:45:00.03 kernel udactivate: error starting virtual disk -1

In this example raw1,raw2,raw3,raw4 will disappear from directory /dev/raw after each restart so will need entries in a system startup script

The commands can be added to file /etc/rc.d/rc.local

#Bind raw devices...

/bin/raw /dev/raw/raw1 /dev/syb_vol_group1/syb_master_lv
/bin/raw /dev/raw/raw2 /dev/syb_vol_group1/syb_sysprocs_lv
/bin/raw /dev/raw/raw3 /dev/syb_vol_group1/syb_data001_lv
/bin/raw /dev/raw/raw4 /dev/syb_vol_group1/syb_log001_lv

#Change permissions on the raw devices...

/bin/chmod 600 /dev/raw/raw1
/bin/chmod 600 /dev/raw/raw2
/bin/chmod 600 /dev/raw/raw3
/bin/chmod 600 /dev/raw/raw4

#Change ownership on the raw devices...

/bin/chown sybase:sybase /dev/raw/raw1
/bin/chown sybase:sybase /dev/raw/raw2
/bin/chown sybase:sybase /dev/raw/raw3
/bin/chown sybase:sybase /dev/raw/raw4

After adding the above commands to rc.local, testing with a system reboot and then starting the sybase ASE dataserver the devices then initialise within sybase as expected:


00:0006:00000:00001:2013/09/06 11:31:51.92 server Activating disk 'RAW_DATA001' of size 512000 KB.
00:0006:00000:00001:2013/09/06 11:31:51.94 kernel Initializing virtual device 12, '/home/sybase/devices/data001' with dsync 'off'.
00:0006:00000:00001:2013/09/06 11:31:51.94 kernel Virtual device 12 started using asynchronous i/o.
00:0006:00000:00001:2013/09/06 11:31:51.94 server Activating disk 'RAW_LOG001' of size 512000 KB.
00:0006:00000:00001:2013/09/06 11:31:51.95 kernel Initializing virtual device 13, '/home/sybase/devices/log001' with dsync 'off'.
00:0006:00000:00001:2013/09/06 11:31:51.95 kernel Virtual device 13 started using asynchronous i/o.

*Note there are alternative locations and methods for set up for the system bind commands above which might be considered in a production environment.
eg Setting up in the /etc/rc3.d/ run level 3 directory.

Viewing device in Sybase ASE using sp_helpdevice

sp_helpdevice shows the device as raw. Attributes of dsync off and directio off are set automatically by disk init if not specified.

1> sp_helpdevice RAW_DATA001
2> go
device_name physical_name description status cntrltype vdevno vpn_low vpn_high
---------------------- -------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ------------ ------------------ ------------ -------------- ----------------
RAW_DATA001 /home/sybase/devices/data001 raw device, special, dsync off, directio off, physical disk, 500.00 MB, Free: 450.00 MB 2 0 12 0 255999
(1 row affected)
dbname size allocated vstart lstart
---------------- -------------------------- -------------------------------------- ------------ ------------
raw_test 50.00 MB Sep 5 2013 5:07PM 0 0

(1 row affected)
(return status = 0)


*Note in comparison a file system device will default on initialisation to "dsync off", "directio on" and show in sp_helpdevice as "file system device".

Speed Test: Sybase Raw Devices versus Sybase Filesystem Devices (using directio)

The above example set up the volume group to use all of the second hard disk.

After this the second hard disk was set up with the first partition for raw devices and the second partition for filesystem.

Sybase devices were set up as raw devices and filesystem devices using directio.

A load database test showed, for the same hard disk, the raw devices to work 4.4 times faster.

An insert test involving a source table within the database inserting into an empty table with two non-clustered indexes had previously been set up prior to a dump of the test database.

The insert test after the databases were loaded showed the raw devices to run 12.1 times faster for that particular test. For the insert test the same database was loaded into the two test databases and the tests were run one after the other straight after the databases were loaded.

A second run of the insert tests showed a ratio of speeds of 5.7. The difference in relative results would probably have been because of data cache having contained some of the data pages.

Acknowledgement

Although I've set up unix raw devices for use with Sybase on other unix systems in the past, this was a first run through on Centos linux.

Jeff Hunter wrote an article which was very helpful and much of the set up documented above is based on this document:

http://searchenterpriselinux.techtarget.com/tip/Installing-Oracle9i-920-on-Red-Hat-Linux-80-using-RAW-devices

Here is a link to Centos documentation on logical volume creation

http://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/LV_create.html





Please feel free to print to file to create a ".pdf", but please do not modify.

Additional contributors and editors are welcome to contact me.


Push Symbol. Push Symbol. Door logos



Alastair Cook email:
cookalastair @ btinternet.com

Last update to this page - 7th Oct 2016
Added screen shot of manual


Push Pull Door