Showing posts with label oracle. Show all posts
Showing posts with label oracle. Show all posts

Tuesday, March 25, 2014

Oracle RAC 11.2 Server Pools


Server Pools

There is nothing unusual with 11.2 server pools, but when it comes to nested pools there are some unexpected glitches. First, it looks like the standard "srvctl" command doesn't support nested server pools at all (as of version 11.2) - if you want a nested pool, you have to modify its attribute with crsctl. After you change a pool with crsctl you won't be even able to read its configuration using srvctl anymore.

Lets create a nested server pool:

$ srvctl add srvpool -g avif -l 1 -u 3 -n "lab241"
$ srvctl status srvpool -g avif -a
 
$ crsctl status serverpool -f

$ srvctl add srvpool -g avifapp -l 1 -u 2
$ srvctl add srvpool -g avifbat -l 1 -u 1 -i 100

$ crsctl modify srvpool ora.avifapp -attr "PARENT_POOLS=ora.avif"
$ crsctl modify srvpool ora.avifapp -attr "IMPORTANCE=200"
$ crsctl modify srvpool ora.avifbat -attr "PARENT_POOLS=ora.avif"
$ crsctl status serverpool ora.avifapp -f
 
$ crsctl modify srvpool ora.avifbat -attr "PARENT_POOLS="
$ crsctl modify srvpool ora.avifapp -attr "PARENT_POOLS="
 
Srvctl seems to be completely confused when you alter the configuration of pools with crsctl. Adding a nested pool to a pool makes it invisible to srvctl, you cannnot even start up an instance using srvctl. When you query the database's config, it shows the database is admin managed - despite the fact previously it was policy managed. So it is all messed up.

Bugs

$ crsctl modify srvpool ora.avifapp -attr "EXCLUSIVE_POOLS=AVIF"
CRS-2596: Modifications to the 'EXCLUSIVE_POOLS' attribute of
          server pools are not supported
CRS-4000: Command Modify failed, or completed with errors.

$ srvctl modify srvpool -g avifapp -i 200
PRKO-3130 : Server pool avifapp is internally managed as part of
            administrator-managed database configuration and
            therefore cannot be modified directly via srvpool object. 
 
Today's comment of MOS (26 March 2014): It is not supported to modify the value of EXCLUSIVE_POOLS to some string.

Resources

  1. Server Pools Attributes (11.2)
  2. A blog article about Server Pools (11.2) with some interesting bug catches.

Monday, February 3, 2014

Oracle TIMESTAMP and TIMEZONE

TIMESTAMP

The "Timestamp" datatype has no concept of timezones, it's basically the same as the "date" dataype but with added precision.


select * from scott.emp
where to_timestamp(hiredate) = TIMESTAMP'2003-04-07 00:00:00 CET';

TIMEZONE

Oracle has 2 datatypes who can store timezones: TimeStamp with Time Zone (TSTZ) and TimeStamp with Local Time Zone (TSLTZ).

TimeStamp with Time Zone (TSTZ) data stores the time and the actual timezone offset/name used for that time at the moment of insert. The stored timezone information can be an offset or named timezone. The date format for TIMESTAMP WITH TIME ZONE is determined by the value of the NLS_TIMESTAMP_TZ_FORMAT parameter.



TimeStamp with Local Time Zone (TSLTZ) data stores internally the time converted to/from the database timezone (see point 3) from the timezone specified at insert/select time.
Note that the data stored in the database is normalized to the database time zone, and the time zone offset is not stored as part of the column data, the current DBTIMZONE is used. When users retrieve the data, Oracle Database returns it in the users' local session time zone from the current DBTIMEZONE.
The date format for TIMESTAMP WITH LOCAL TIME ZONE is determined by the value of the NLS_TIMESTAMP_FORMAT parameter. TIMESTAMP WITH LOCAL TIME ZONE data is NOT returned with a timezone in the result set.

If you store TSLTZ data the database timezone should always be an offset.

Note that the SESSIONTIMEZONE (and NOT the database timezone) is the actual used timezone at both insert and select (if not specified explicit) to calculate the time for the inserted/returned result.

 

What is the difference between a timezone offset and a named timezone?

A timezone expressed in a offset (example: -04:00) is always the same and gives the difference compared to UTC (Coordinated Universal Time), practically (although not correct technically speaking) UTC is the same as GMT (Greenwich mean time), and UTC/GMT corresponds to a offset of +00:00.

Because a offset is fixed it cannot change reflect DST (Daylight Saving Time) changes. DST itself is a change of the offset.

A named timezone is a name (example: Canada/Eastern or GMT) who reflects a geographic region or location and this will be mapped to a certain offset for a certain time on a certain date. The offset may or may not change for DST or have been changed during history.

The named timezone Europe/London for example will have a different offset during summer (+01:00) as during winter (+00:00) and Oracle has the definitions on when this DST change happens stored, so it can adjust it at the right date and time.

The named timezone UTC for example is always +00:00.

Database Timezone (dbtimezone)

Below excerpts from MOS Article 340512.1:

The database time zone is not as important as it sounds. First of all it does not influence functions like SYSDATE, or SYSTIMESTAMP. These function take their contents (date and time, and in the case of SYSTIMESTAMP also time zone) completely from the OS without any "Oracle" intervention.
The only function of the database time zone is that it functions as a time zone in which the values of the "TIMESTAMP WITH LOCAL TIME ZONE" (TSLTZ) datatype are normalized to the current database timezone when they are stored in the database. However, these stored values are always converted into the session time zone on insert and retrieval, so the actual setting of the database time zone is more or less immaterial. The dbtimezone should be set to an offset (+00:00 , -05:00 or +09:00 for example) or a static time zone that is not affected by DST (like UTC or GMT ).
A common misconception is that the database timezone needs to be "your" timezone. This is NOT true. The database timezone has NO relation with "where" the server is located.
There is NO advantage whatsoever in using your timezone or a named timezone as database timezone.
The best setting for dbtimezone is simply +00:00 (or any other OFFSET like -09:00, +08:00, ...), if your current dbtimezone value is an OFFSET then please leave it like it is. The database time zone is only used as a time zone in which stored TSLTZ values are normalized.
So the value of the dbtimezone should in fact not change. Because this is the only task for the database time zone it should not be used for any other things.

The database time zone is usually only set at creation time of the database:
SQL> CREATE DATABASE...
     SET TIME_ZONE='+00:00';
This will only work if there are no TSLTZ values already stored in the database or an ORA-02231 (9i) or ORA-30079 will be seen 

SYSTIMESTAMP

You could say that SYSTIMESTAMP is "SYSDATE with time zone information added".
SYSTIMESTAMP, just like SYSDATE depends on Unix platforms on the UNIX time configuration (= Unix TZ variable) for the Unix session when the database and listener where started.
The precision is platform dependant, on most Unix platforms it's microseconds (10-6) on Windows this is Milliseconds (10-3). The output is defined by NLS_TIMESTAMP_TZ_FORMAT in NLS_SESSION_PARAMETERS.

The SYSTIMESTAMP output has an offset from UTC but is not defined to include an actual named timezone. Mapping this offset to a timezone, or reinterpreting the OS TZ setting would be very hard and error prone and so the code sticks to the absolute offset between UTC and the local time.
You can see this by issuing:

SQL> SELECT EXTRACT( timezone_region from systimestamp ) FROM dual;
EXTRACT(TIMEZONE_REGIONFROMSYSTIMESTAMP)
----------------------------------------------------------------
UNKNOWN

if you want to know the time in a particular time zone you can use "AT TIME ZONE".
SQL> SELECT systimestamp AT TIME ZONE 'Canada/Eastern' FROM DUAL;
SYSTIMESTAMPATTIMEZONE'CANADA/EASTERN'
------------------------------------------------------------------
19-NOV-07 09.47.55.099000 CANADA/EASTERN

if you are wanting to see the timestamp in the sessions timezone then you should use CURRENT_TIMESTAMP.

SQL> ALTER SESSION SET NLS_TIMESTAMP_TZ_FORMAT ='YYYY.MM.DD HH24:MI:SS TZR TZD';
Session altered.
SQL> ALTER SESSION SET TIME_ZONE = 'Canada/Eastern';
Session altered.
SQL> SELECT current_timestamp FROM dual;
CURRENT_TIMESTAMP
---------------------------------------------------------------------------
2007.11.19 09:48:18 CANADA/EASTERN EST

Please note that the DBTIMEZONE has NOTHING to do with this, changing DBTIMEZONE will NOT solve SYSDATE or SYSTIMESTAMP returning a wrong time.

Difference between CURRENT_DATE, LOCALTIMESTAMP and CURRENT_TIMESTAMP?

They all depend on the session timezone, which is defined on the CLIENT side, not server side.

CURRENT_DATE returns the current date and time in the session time zone in a value of datatype DATE.

LOCALTIMESTAMP returns the current date and time in the session time zone in a value of datatype TIMESTAMP.

CURRENT_TIMESTAMP returns the current date and time in the session time zone, in a value of datatype TIMESTAMP WITH TIME ZONE.

The sessions NLS_DATE_FORMAT defines the output format of a DATE, NLS_TIMESTAMP_FORMAT defines the output format of a TIMESTAMP, the NLS_TIMESTAMP_TZ_FORMAT defines the output format of a TIMESTAMP WITH TIME ZONE.



How can I check the session time zone?

The SESSIONTIMEZONE sql function returns the value of the current session's time zone:

SQL> SELECT SESSIONTIMEZONE FROM DUAL;
SESSIONTIMEZONE
---------------
+01:00

How can I set the session time zone?

By default the session timezone is set to the OFFSET of the clients (!) operating system timezone value/setting at connection time. The client asks the client Os what the current offset is to UTC and then , during the connection/session creation fase , does an alter session based on this.
The session time zone can be explicitly set to:
  • O/S local time zone
  • Database time zone
  • An absolute offset
  • A named region
This can be done in 2 ways, the first method consists to use one of the following ALTER SESSION SET TIME_ZONE statements:
SQL> ALTER SESSION SET TIME_ZONE = local;
SQL> ALTER SESSION SET TIME_ZONE = dbtimezone;
SQL> ALTER SESSION SET TIME_ZONE = '-05:00';
SQL> ALTER SESSION SET TIME_ZONE = 'Europe/London';
The alternative method is to set the (client) operating system environment variable ORA_SDTZ:
ORA_SDTZ = 'OS_TZ' | 'DB_TZ' | '[+ | -] hh:mm' | 'timezone_region'
$ ORA_SDTZ='OS_TZ'
$ export ORA_SDTZ
$ ORA_SDTZ='-05:00'
$ export ORA_SDTZ
If you do not want to set ORA_SDTZ on all client machines, but still want to be in control over the session time zone settings for all sessions, then consider using a logon trigger to the database, in which you can set the session time zone specifically through ALTER SESSION (as per point a above).
Note that the session timezone defaults to an offset ( like +05:00), even if the unix TZ variable or Windows timezone region is set to a named TZ. If you need the session timezone to be a named timezone then you need to set ORA_SDTZ client (!) environment (or registry on windows) with an Oracle TZ name.

How can I retrieve the time zone offset corresponding to a time zone region?

The TZ_OFFSET() sql function returns the time zone offset displacement to the input time zone region.

SQL> SELECT TZ_OFFSET('US/Pacific') FROM DUAL;

TZ_OFFS
-------
-07:00
The returned offset depends on the date this statement is executed. For example, in the 'US/Pacific' time zone, it may return '-07:00' or '-08:00' whether daylight saving is in effect or not.



Other Resources


To see a listing of valid time zone region names, query the TZNAME column of the V$TIMEZONE_NAMES dynamic performance view.

SELECT SESSIONTIMEZONE FROM DUAL; http://docs.oracle.com/cd/E11882_01/server.112/e26088/functions162.htm#SQLRF51736


ALTER SESSION SET
TIME_ZONE =  '[+ | -] hh:mm' 
             | LOCAL 
             | DBTIMEZONE 
             | 'time_zone_region'
 MOS:
  1. Briefing by Jonathan Lewis, particularly interesting when considering a column type TIMESTAMP WITH LOCAL TIME ZONE  
  2. 340512.1 Timestamps & time zones – Frequently Asked Questions
  3. ALTER SESSION Reference.

Friday, January 10, 2014

JDBC Connection Failover

FAN

With JDBC thin 11.2 and simplefan.jar (oracle.simplefan package) I was able to perform tests only with ONS client running locally. The program and the test environment has been described at Martin Bach's blog. Ons can be installed with the full version of Oracle 11.2 Windows client (standard install). It requires changes in the ons.config: usesharedinstall=false and nodes=node1:6200,node2:6200,node3:6200. By default usessharedinstall is set to true, which makes the lock file appending the hostname to it. Consequently it renders the example being not able to found it by the standard name. I didn't investigate how to fix it at the code level.

Speaking on JDBC thin it is suprising how little of information I could find in the Internet. The only valuable information from Oracle comes with Oracle Database RAC FAN Events Java 11g Release 2 (11.2) E13993-01 and there is not working example in Oracle Database JDBC Developer's Guide 11g Release 2 (11.2). Not working not only because it is not quite complete, but because it refers to not existing methods: Actually didn't find methods like getServiceMemberStatus, getServiceCompositeEvent and some others within simplefan.jar and the above API documentation. Searching with Google for these methods returns the above (not working) example only. In a similar way, existing function don't handle coming events properly. The "SHUTDOWN" event is an example here.

The code below is original Oracle's example mixed with Martin Bach's code:

package org.kedra.fan1;
import java.util.Properties;
import oracle.simplefan.FanEventListener;
import oracle.simplefan.FanManager;
import oracle.simplefan.FanSubscription;
import oracle.simplefan.LoadAdvisoryEvent;
import oracle.simplefan.NodeDownEvent;
import oracle.simplefan.ServiceDownEvent;
import oracle.simplefan.ServiceDownEvent.ServiceMemberEvent;

public class Test1 {
 Test1() {
  FanManager fm = FanManager.getInstance();

  //Properties fmProps = new Properties();
  //fmProps.setProperty("onsNodes",
  //  "plabb241:6200,plabb230:6200,plabb229:6200");
  // fmProps.setProperty(key, value)
  // fm.configure(fmProps);

  System.setProperty("oracle.ons.oraclehome",
    "c:\\Oracle2\\product\\11.2.0\\client_1");
  System.out.println(System.getProperty("oracle.ons.oraclehome"));

  Properties props = new Properties();
  props.put("serviceName", "AMQS1");

  FanSubscription sub = fm.subscribe(props);
  System.out.println("I'm subscribed!");

  sub.addListener(new FanEventListener() {

   public void handleEvent(ServiceDownEvent event) {
    System.out.println("ServiceDownEvent!");

    try {
     System.out.println(event.getTimestamp());
     System.out.println("getServiceName: "
       + event.getServiceName());
     System.out.println("getDatabaseUniqueName: "
       + event.getDatabaseUniqueName());
     System.out.println("getReason: " + event.getReason());
     System.out.println("getKind: " + event.getKind());

     ServiceMemberEvent me = event.getServiceMemberEvent();
     if (me != null) {
      System.out.println("getServiceMemberEvent");
      System.out.println("\t getInstanceName: "
        + me.getInstanceName());
      System.out.println("\t getNodeName: "
        + me.getNodeName());
      // System.out.println(me.getServiceMemberStatus());
     }
     // ServiceCompositeEvent ce = se.getServiceCompositeEvent();
     // if (ce != null) {
     // System.out.println(ce.getServiceCompositeStatus());
     // }
    } catch (Throwable t) {
     // handle all exceptions and errors
     System.err.println("handleEvent - ServiceDownEvent");
     t.printStackTrace(System.err);
    }
   }

   public void handleEvent(NodeDownEvent event) {
    System.out.println("Node Down Event!");

    try {
     System.out.println(event.getTimestamp());
     System.out.println("getNodeName: " + event.getNodeName());
     // ServiceCompositeEvent ce = se.getServiceCompositeEvent();
     // if (ce != null) {
     // System.out.println(ce.getServiceCompositeStatus());
     // }
    } catch (Throwable t) {
     System.err.println("handleEvent - ServiceDownEvent");
     t.printStackTrace(System.err);
    }
   }

   public void handleEvent(LoadAdvisoryEvent arg0) {
    System.out.println("Load Advisory event");

    System.out.println("originating database: "
      + arg0.getDatabaseUniqueName());
    System.out.println("originating instance: "
      + arg0.getInstanceName());
    System.out.println("Service Quality     : "
      + arg0.getServiceQuality());
    System.out.println("Percent             : " + arg0.getPercent());
    System.out.println("Service Name        : "
      + arg0.getServiceName());
    System.out.println("Service Quality     : "
      + arg0.getServiceQuality());
    System.out.println("Observed at         : "
      + arg0.getTimestamp() + "\n\n");
   }
  });
 }

 public static void main(String[] args) {
  Test1 tc = new Test1();

  int i = 0;
  while (i < 100000) {
   try {
    Thread.sleep(100);
    i++;
   } catch (Exception e) {
    System.out.println(e);
   }
  }

  System.out.println("execution ended");
 }
}

I would like to see a support for a remote ONS. There is a method configure() in the FanManager class which takes a list of ONS servers but requires mandatory oracle wallet and password as well. This is not something I expect - for I don't want the communication with ONS server over SSL.

Oracle admits it is "the initial release" so the functionality of this library is fairly limited.

Fast Connection Failover

Fast Connection Failover (FCF) and for JDBC thin it us usable with Universal Connection Pool (UCP). There is an option to use JDBC Implicit Connection Cache (ICC) but Oracle considers the feature to be replaced by UCP.

  1. Automatic Workload Management with Oracle Real Application Clusters 11g Release 2 - FAN, FCF
  2. Maximum Availability Architecture, Oracle Best Practices For High Availability: Client Failover Best Practices for Highly Available Oracle  Databases: Oracle Database 11g Release 2 - Standby database oriented.
  3. Fast Connection Failover Example JDBC Thin - this is about 10g High Availability and JDBC. Requires ons.jar or ONS working on the client side. Great resource for hands-on experience.
  4. How to Verify Universal Connection Pool (UCP) / Fast Connection Failover (FCF) Setup (Metalink Doc ID 1064652.1)
  5. JDBC and Universal Connection Pool
  6. UCP Demos Page (Oracle)
  7. TAF vs FAN, FCF vs ONS
  8.  Metalink Resources:


    Monday, December 30, 2013

    Oracle's New Features

    Top New Features As Used by Me

    Subquery Factoring aka WITH clause

    What a surprise such the useful clause has been ignored by me until now. And it is here since 9i. I found it in a book "Cost Based Oracle: Fundamentals" by Jonathan Lewis. There is an in-depth article at Oracle-Base about the feature.

    Virtual Columns

    The virtual column is an extra column in a table which can be a result of calculation based on existing real columns. It came with Oracle 11g and simplified queries very much. It can be used for table partitioning,

    Tuesday, October 15, 2013

    FGA Notes

    FGA Notes @11G


    To create a fine-grained audit policy, you must have EXECUTE privileges on the DBMS_FGA PL/SQL package. The package is owned by the SYS user.

    To create a fine-grained audit policy, use the DBMS_FGA.ADD_POLICY procedure. This procedure creates an audit policy using the supplied predicate as the audit condition. Oracle Database executes the policy predicate with the privileges of the user who created the policy. The maximum number of fine-grained policies on any table or view object is 256. Oracle Database stores the policy in the data dictionary table, but you can create the policy on any table or view that is not in the SYS schema.
    After you create the fine-grained audit policy, it does not reside in any specific schema, although the definition for the policy is stored in the SYS.FGA$ data dictionary table.
    You cannot modify a fine-grained audit policy after you have created it. If you need to modify the policy, drop it and then recreate it.

    DBMS_FGA.ADD_POLICY(
       object_schema      VARCHAR2, 
       object_name        VARCHAR2, 
       policy_name        VARCHAR2, 
       audit_condition    VARCHAR2, 
       audit_column       VARCHAR2, 
       handler_schema     VARCHAR2, 
       handler_module     VARCHAR2, 
       enable             BOOLEAN, 
       statement_types    VARCHAR2,
       audit_trail        BINARY_INTEGER IN DEFAULT,
       audit_column_opts  BINARY_INTEGER IN DEFAULT);
     
    In this specification:
    • object_schema: Specifies the schema of the object to be audited. (If NULL, the current log-on user schema is assumed.)
    • object_name: Specifies the name of the object to be audited.
    • policy_name: Specifies the name of the policy to be created. Ensure that this name is unique.
    • audit_condition: Specifies a Boolean condition in a row. NULL is allowed and acts as TRUE. See "Auditing Specific Columns and Rows" for more information. If you specify NULL or no audit condition, then any action on a table with that policy creates an audit record, whether or not rows are returned
    • audit_column: Specifies one or more columns to audit, including hidden columns. If set to NULL or omitted, all columns are audited. These can include Oracle Label Security hidden columns or object type columns. The default, NULL, causes audit if any column is accessed or affected.
    • handler_schema: If an alert is used to trigger a response when the policy is violated, specifies the name of the schema that contains the event handler. The default, NULL, uses the current schema. See also "Tutorial: Adding an Email Alert to a Fine-Grained Audit Policy".
    • handler_module: Specifies the name of the event handler. Include the package the event handler is in. This function is invoked only after the first row that matches the audit condition in the query is processed.
      Follow these guidelines:
      • Do not create recursive fine-grained audit handlers. For example, suppose you create a handler that executes an INSERT statement on the HR.EMPLOYEES table. The policy that is associated with this handler is for INSERT statements (as set by the statement_types parameter). When the policy is used, the handler executes recursively until the system has run out of memory. This can raise the error ORA-1000: maximum open cursors exceeded or ORA-00036: maximum number of recursive SQL levels (50) exceeded.
      • Do not issue the DBMS_FGA.ENABLE_POLICY or DBMS_FGA.DISABLE_POLICY statement from a policy handler. Doing so can raise the ORA-28144: Failed to execute fine-grained audit handler error.
    • enable: Enables or disables the policy using true or false. If omitted, the policy is enabled. The default is TRUE.
    • statement_types: Specifies the SQL statements to be audited: INSERT, UPDATE, DELETE, or SELECT only.
    • audit_trail: Specifies the destination (DB or XML) of fine-grained audit records. Also specifies whether to populate LSQLTEXT and LSQLBIND in FGA_LOG$. However, be aware that sensitive data, such as credit card information, can be recorded in clear text. See "Auditing Sensitive Information" for how you can handle this scenario.
      If you set the audit_trail parameter to XML, then the XML files are written to the directory specified by the AUDIT_FILE_DEST initialization parameter.
      For read-only databases, Oracle Database writes the fine-grained audit trail to XML files, regardless of the audit_trail setting.
    • audit_column_opts: If you specify more than one column in the audit_column parameter, then this parameter determines whether to audit all or specific columns. See "Auditing Specific Columns and Rows" for more information.
    BEGIN
      DBMS_FGA.ADD_POLICY(
       object_schema      => 'HR',
       object_name        => 'EMPLOYEES',
       policy_name        => 'chk_hr_employees',
       enable             =>  TRUE,
       statement_types    => 'INSERT, UPDATE, SELECT, DELETE',
       audit_trail        =>  DBMS_FGA.DB+DBMS_FGA.EXTENDED);
    END;
    /
    
    DBMS_FGA.DISABLE_POLICY(
      object_schema        => 'HR',
      object_name          => 'EMPLOYEES',
      policy_name          => 'chk_hr_employees');
    
    DBMS_FGA.DROP_POLICY(
      object_schema      => 'HR',
      object_name        => 'EMPLOYEES',
      policy_name        => 'chk_hr_employees'); 
     
    SELECT * FROM DBA_AUDIT_POLICIES; 
     
     
    SYS@DEV1> show parameter audit_trail;
    NAME                                 TYPE        VALUE
    ------------------------------------ ----------- ------------------------------
    audit_trail                          string      DB 
     
    SYS@DEV1> SET SERVEROUTPUT ON
    BEGIN
    IF
    DBMS_AUDIT_MGMT.IS_CLEANUP_INITIALIZED(
               DBMS_AUDIT_MGMT.AUDIT_TRAIL_AUD_STD)
      THEN
         DBMS_OUTPUT.PUT_LINE('AUD$ is initialized for cleanup');
      ELSE
         DBMS_OUTPUT.PUT_LINE('AUD$ is not initialized for cleanup.');
      END IF;
    END;
    SYS@DEV1>
    AUD$ is not initialized for cleanup.
    PL/SQL procedure successfully completed.
    SYS@DEV1>
     
    Here is an excerpt form the documentation: DEFAULT_CLEANUP_INTERVAL: Specify the desired default hourly purge interval (for example, 12 for every 12 hours). The DBMS_AUDIT_MGMT procedures use this value to determine how to purge audit records. The timing begins when you run the DBMS_AUDIT_MGMT.INIT_CLEANUP procedure. To update this value later, set the DBMS_AUDIT_MGMT.CLEAN_UP_INTERVAL property of the DBMS_AUDIT_MGMT.SET_AUDIT_TRAIL_PROPERTY procedure. The DEFAULT_CLEANUP_INTERVAL setting must indicate the frequency in which DBMS_AUDIT_MGMT.CLEAN_AUDIT_TRAIL is called. If you are uncertain about the frequency, set it to an approximate value. You can change this value later on by using the DBMS_AUDIT_MGMT.SET_AUDIT_TRAIL_PROPERTY procedure.

    BEGIN
     DBMS_AUDIT_MGMT.INIT_CLEANUP(
      AUDIT_TRAIL_TYPE            => DBMS_AUDIT_MGMT.AUDIT_TRAIL_ALL,
      DEFAULT_CLEANUP_INTERVAL    => 48 );
    END;
    /
     
    You can cancel the DBMS_AUDIT_MGMT.INIT_CLEANUP settings, that is, the default cleanup interval, by invoking the DBMS_AUDIT_MGMT.DEINIT_CLEANUP procedure. For example, to cancel all purge settings for the standard audit trail:

    BEGIN
     DBMS_AUDIT_MGMT.DEINIT_CLEANUP(
      AUDIT_TRAIL_TYPE  => DBMS_AUDIT_MGMT.AUDIT_TRAIL_AUD_STD);
    END;
    /

    Views:

    SELECT * FROM DBA_AUDIT_POLICIES;

    SELECT * FROM DBA_FGA_AUDIT_TRAIL;
    SELECT * FROM DBA_AUDIT_MGMT_CLEAN_EVENTS;
    SELECT * FROM DBA_AUDIT_MGMT_CLEANUP_JOBS;
    SELECT * FROM DBA_AUDIT_MGMT_CONFIG_PARAMS;
    SELECT * FROM DBA_AUDIT_MGMT_LAST_ARCH_TS;

    Documentation Pointers

    1. Oracle® Database Security Guide 11g Release 2 (11.2)

    Tuesday, September 10, 2013

    Oracle 11G Auditing

    Password change for the application user is the main reason I want auditing now. I have an user called PASGEN, password has been changed but still some application processes exit which try to connect using the wrong password. Existing profiles make the account immediately locked so I had to (temporarily) alter the profile and make failed_login_attempts unlimited:

    ALTER PROFILE nonexpiring LIMIT failed_login_attempts UNLIMITED;

    After that I setup an audit to find out whats the issue:

    AUDIT SESSION BY PASGEN BY ACCESS WHENEVER NOT SUCCESSFUL;

    Then I had an hour or longer wasted trying to find out why I see wrong records from SYS.AUD$. I tried to filter only new entries using ntimestamp# - for unknown reason timestamp# (type DATE) has not been populated. Anway - ntimestamp# appears to be in UTC timezone and it took me some time to figure it out. Finally I found DBA_AUDIT_TRAIL and this is where I should start.

    So query which work best for me is:

    SELECT os_username,username,userhost,os_process,terminal, action_name,comment_text, timestamp
        FROM dba_audit_trail
    WHERE returncode = 1017
        AND timestamp > localtimestamp - 1/24/60 * 5 -- last 5 minutes
        AND username='PASGEN'
    ORDER BY timestamp, os_username,userhost,os_process;


    And sometimes - to see more details - I put asterisk instead of list of columns for SELECT above.

    Another interesting issue with the database based audit trail (standard audit trail) is this log grows quickly if there is a lot of entries to be stored. That's particularly true statement when you have an application trying to connect with the wrong password over and over, sometimes a couple of times per second. So the SYSAUX tablespace may be overflowed quickly. This is the reason I implemented automatic cleanup for this tablespace:



    SYS@DEV1> show parameter audit_trail;
    NAME TYPE   VALUE

    ----------- ------------------------------
    audit_trail string DB
    SYS@DEV1>
    SET SERVEROUTPUT ON
    BEGIN
      IF
        DBMS_AUDIT_MGMT.IS_CLEANUP_INITIALIZED(
        DBMS_AUDIT_MGMT.AUDIT_TRAIL_AUD_STD)
      THEN
        DBMS_OUTPUT.PUT_LINE('AUD$ is initialized for cleanup');
      ELSE
        DBMS_OUTPUT.PUT_LINE('AUD$ is not initialized for cleanup.');
      END IF;
    END;
    /
    SYS@DEV1>
    AUD$ is not initialized for cleanup.
    PL/SQL procedure successfully completed.
    SYS@DEV1>


    Here is an excerpt form the documentation:

    DEFAULT_CLEANUP_INTERVAL: Specify the desired default hourly purge interval (for example, 12 for every 12 hours). The DBMS_AUDIT_MGMT procedures use this value to determine how to purge audit records. The timing begins when you run the DBMS_AUDIT_MGMT.INIT_CLEANUP procedure. To update this value later, set the DBMS_AUDIT_MGMT.CLEAN_UP_INTERVAL property of the DBMS_AUDIT_MGMT.SET_AUDIT_TRAIL_PROPERTY procedure.

    The DEFAULT_CLEANUP_INTERVAL setting must indicate the frequency in which DBMS_AUDIT_MGMT.CLEAN_AUDIT_TRAIL is called. If you are uncertain about the frequency, set it to an approximate value. You can change this value later on by using the DBMS_AUDIT_MGMT.SET_AUDIT_TRAIL_PROPERTY procedure.

    BEGIN
     DBMS_AUDIT_MGMT.INIT_CLEANUP(
      AUDIT_TRAIL_TYPE            => DBMS_AUDIT_MGMT.AUDIT_TRAIL_ALL,
      DEFAULT_CLEANUP_INTERVAL    => 48 );
    END;
    /
     

    You can cancel the DBMS_AUDIT_MGMT.INIT_CLEANUP settings, that is, the default cleanup interval, by invoking the DBMS_AUDIT_MGMT.DEINIT_CLEANUP procedure.
    For example, to cancel all purge settings for the standard audit trail:

    BEGIN
     DBMS_AUDIT_MGMT.DEINIT_CLEANUP(
      AUDIT_TRAIL_TYPE  => DBMS_AUDIT_MGMT.AUDIT_TRAIL_AUD_STD);
    END;
    /

    Views


    SELECT * FROM DBA_FGA_AUDIT_TRAIL;
    SELECT * FROM DBA_AUDIT_MGMT_CLEAN_EVENTS;
    SELECT * FROM DBA_AUDIT_MGMT_CLEANUP_JOBS;
    SELECT * FROM DBA_AUDIT_MGMT_CONFIG_PARAMS;
    SELECT * FROM DBA_AUDIT_MGMT_LAST_ARCH_TS;

     

    Documentation Pointers

    1. Oracle® Database Security Guide 11g Release 2 (11.2)



    Tuesday, August 6, 2013

    Statistics in 11.2

    Collecting Stats Faster

    The problem is collecting optimizer statistics with DBMS_STATS.GATHER_SCHEMA_STATS happens serially and takes incredibly long time for huge databases. Particularly when we have LOB tables. Usually I need rebuild statistics after a schema is imported from previous version of the database to the latest one. So I don't care what load it brings to the database - it might be very intense operation but I want it to finish as soon as possible. And usually the load incurred by the usual GATHER_SCHEMA_STATS procedure is minimal so making it running in parallel would be an excellent idea.

    One night, when statistics collection already taken a long time, I decided to terminated and go this way:

    BEGIN
       DBMS_STATS.GATHER_SYSTEM_STATS('INTERVAL', 180); 
       DBMS_STATS.SET_GLOBAL_PREFS('CONCURRENT','TRUE');
       DBMS_STATS.GATHER_SCHEMA_STATS( 'MYUSER',
            degree  => DBMS_STATS.AUTO_DEGREE,
            options => 'GATHER',
            cascade => TRUE,
            no_invalidate => FALSE
    );
    END;
    /

    The effect was terrific. Mainly because it depends on the parameter job_queue_processes and it was set to 1000. So the database was really under a load for some time but the process finished in 30 minutes. I don't think the job_queue_processed is set to a reasonable parameter here for daily operation but at least it worked for this case!
    I specified options=>'GATHER' is required when you want to collects statistics for all objects in the schema. With the default 'GATHER AUTO' Oracle implicitly determines which objects need new statistics, and determines how to gather those statistics. When GATHER AUTO is specified, the only additional valid parameters are ownname, stattab, statid, objlist and statown; all other parameter settings are ignored. Returns a list of processed objects.

    Checking the current status of concurrent statistics collection:

    SELECT DBMS_STATS.GET_PREFS('CONCURRENT') pref FROM dual;
    
    

    Resources

    1. GATHER_SCHEMA_STATS in 11.2
    2. Concurrent Statistics Gathering.
    3. Understanding Optimizer Statistics (PDF).

    Tuesday, July 30, 2013

    Oracle XDB

    What XDB database is? I didn't use it until recently when I had to assign some network privileges (see my other post on Fine Grained Access). And today I found a database where DBMS_NETWORK_ACL package is not installed. Google redirects me to information about XDB database missing. My first try failed, unfortunately I didn't spool the output to a file. It was because a dedicated XDB tablespace was missing - the script doesn't handle that automatically. Fortunately there is a lot of resources on Internet how to drop and recreate XDB:

    @?/rdbms/admin/catnoqm.sql
    drop trigger sys.xdb_installation_trigger;
    drop trigger sys.dropped_xdb_instll_trigger;
    drop table dropped_xdb_instll_tab;


    CREATE tablespace XDB datafile '+DATA' size 10M autoextend on next 10m maxsize unlimited;

    SPOOL /tmp/xdb.log REPLACE
    @?/rdbms/admin/catqm.sql xdbpasswd XDB TEMPSPACE NO
    SPOOL OFF


    So far I have no idea what less for XDB is actually used.

    External Links


    http://www.dbatools.net/experience/oracle_xmldb_install.html
    http://www.oracle-wiki.net/startdocshowtoinstallxmldb

    Monday, July 22, 2013

    Oracle 11g Statistics Collection (AUTOTASK)



          I noticed this subject comes back over and over: no statistics gathering job seems to be activated. Well, usually it is enabled until something unexpected has been done in the database. At 11g gathering optimizer stats it is enabled by default. It doesn’t exist (as in 10g) as a dedicated job anymore. Right now it is maintained using DBMS_AUTOTASK_ADMIN_PACKAGE. When I run a query checking the state of “auto optimizer stats collection” I can see it running every day.


    SELECT * FROM DBA_AUTOTASK_JOB_HISTORY
    WHERE client_name = 'auto optimizer stats collection'
    ORDER BY job_start_time DESC;

    Wednesday, July 17, 2013

    Fine-Grained Access in PL/SQL Packages




       With Oracle 11g there is a new set or privileges. You can configure user access control to external network services and wallets through the UTL_TCP, UTL_SMTP, UTL_MAIL, UTL_HTTP, and UTL_INADDR PL/SQL packages, the DBMS_LDAP PL/SQL package, and the HttpUriType type.

    To configure fine-grained access control to external network services, you create an access control list (ACL), which is stored in Oracle XML DB. You can create the access control list by using Oracle XML DB itself, or by using the DBMS_NETWORK_ACL_ADMIN and DBMS_NETWORK_ACL_UTILITY PL/SQL packages.

    This feature enhances security for network connections because it restricts the external network hosts that a database user can connect to using the PL/SQL network utility packages UTL_TCP, UTL_SMTP, UTL_MAIL, UTL_HTTP, and UTL_INADDR, the DBMS_LDAP PL/SQL package, and the HttpUriType type. Otherwise, an intruder who gained access to the database could maliciously attack the network, because, by default, the PL/SQL utility packages are created with the EXECUTE privilege granted to PUBLIC users.

    If you have upgraded from a release before Oracle Database 11g Release 1 (11.1), and your applications depend on PL/SQL network utility packages UTL_TCP, UTL_SMTP, UTL_MAIL, UTL_HTTP, and UTL_INADDR, the DBMS_LDAP PL/SQL package, or the HttpUriType type, then the following error may occur when you try to run the application:
    ORA-24247: network access denied by access control list (ACL) 
     
    You cannot import or export the access control list settings
    by using the Oracle Database import or export utilities
    such as Oracle Data Pump.
     
     Use the DBMS_NETWORK_ACL_ADMIN.CREATE_ACL procedure to create the content of the access control list. It contains a name of the access control list, a brief description, and privilege settings for one user or role that you want to associate with the access control list. In an access control list, privileges for each user or role are grouped together as an access control entry (ACE). An access control list must have the privilege settings for at least one user or role.

    BEGIN
     DBMS_NETWORK_ACL_ADMIN.CREATE_ACL (
      acl          => 'file_name.xml', 
      description  => 'file description',
      principal    => 'user_or_role',
      is_grant     => TRUE|FALSE, 
      privilege    => 'connect|resolve',
      start_date   => null|timestamp_with_time_zone,
      end_date     => null|timestamp_with_time_zone); 
    END;
     
    In this specification:
    • acl: name for the access control list XML file. Oracle Database creates this file relative to the /sys/acls directory in the XML DB Repository in the database. Include the .xml extension. 
    • principal: User account or role being granted or denied permissions. User account or role in case sensitive characters, entering it in mixed or lower case will not work.
      If you want to enter multiple users or grant additional privileges to this user or role, use the DBMS_NETWORK_ACL.ADD_PRIVILEGE procedure.
    • is_grant: Either TRUE or FALSE, to indicate whether the privilege is to be granted or denied.
    • privilege: Enter either connect or resolve. Case sensitive, always it in lowercase. The connect privilege grants the user permission to connect to a network service at an external host. The resolve privilege grants the user permission to resolve a network host name or an IP address.
      A database user needs the connect privilege to an external network host computer if he or she is connecting using the UTL_TCP, UTL_SMTP, UTL_MAIL, UTL_HTTP, the DBMS_LDAP package, and the HttpUriType type. To resolve the host name that was given a host IP address, or the IP address that was given a host name, with the UTL_INADDR package, grant the database user the resolve privilege instead.
    • start_date: (Optional) Enter the start date for the access control entry (ACE), in TIMESTAMP WITH TIME ZONE format (YYYY-MM-DD HH:MI:SS.FF TZR). When specified, the access control entry will be valid only on or after the specified date. The default is null. For example, to set a start date of February 28, 2008, at 6:30 a.m. in San Francisco, California, U.S., which is in the Pacific time zone:
      start_date => '2008-02-28 06:30:00.00 US/Pacific',
      
      The NLS_TIMESTAMP_FORMAT initialization parameter sets the default timestamp format.
    • end_date: (Optional) Enter the end date for the ACE.
    BEGIN
     DBMS_NETWORK_ACL_ADMIN.ADD_PRIVILEGE ( 
      acl         => 'file_name.xml', 
      principal   => 'user_or_role',
      is_grant    => TRUE|FALSE, 
      privilege   => 'connect|resolve', 
      position    => null|value, 
      start_date  => null|timestamp_with_time_zone,
      end_date    => null|timestamp_with_time_zone);
    END;
     
    You can grant the privilege explicitly or you can grant it to a role.

    After you create the access control list, then you are ready to assign it to one or more network host computers. You can use the DBMS_NETWORK_ACL_ADMIN.ASSIGN_ACL procedure to do so.
    For example:
    BEGIN
     DBMS_NETWORK_ACL_ADMIN.ASSIGN_ACL (
      acl         => 'file_name.xml',
      host        => 'network_host', 
      lower_port  => null|port_number,
      upper_port  => null|port_number); 
    END;
    
    In this specification:
    • acl: the name of the access control list XML file to assign to the network host. Oracle Database creates this file relative to the /sys/acls directory in the XML DB Repository in the database.
    • host: the network host to which this access control list will be assigned. This setting can be a name or IP address of the network host. Host names are case insensitive. If you specify localhost, and if the host name has not been specified with the UTL_INADDR and UTL_HTTP PL/SQL packages in situations in which the local host is assumed, then these packages will search for and use the ACL that has been assigned localhost for the host setting.
    • lower/upper_port: (Optional) For TCP connections, enter the lower boundary of the port range. Use this setting for the connect privilege only; omit it for the resolve privilege. The default is null, which means that there is no port restriction (that is, the ACL applies to all ports). The range of port numbers is between 1 and 65535.
       
    Only one access control list can be assigned to any host computer, domain, or IP subnet, and if specified, the TCP port range. When you assign a new access control list to a network target, Oracle unassigns the previous access control list that was assigned to the same target. However, Oracle does not drop the access control list. You can drop the access control list by using the DROP_ACL procedure. To remove an access control list assignment, use the UNASSIGN_ACL procedure.


    All access control list changes, including the assignment to network hosts, are transactional. They do not take effect until the transaction is committed.


    Example

    Assuming we want to give an access to PL/SQL package in schema SCOTT for sending out emails using specific mail gateway named mail.acme.org, no time constraints. Because it is for SMPT access, we are interested in accessing the port number 25.



    DECLARE
      acl_already_exists    EXCEPTION;
      PRAGMA EXCEPTION_INIT (acl_already_exists, -31003);
    BEGIN
        BEGIN
            DBMS_NETWORK_ACL_ADMIN.CREATE_ACL (
            acl          => 'sendmail.xml',
            description  => 'Permissions for sending report emails',
            principal    => 'SCOTT',
            is_grant     => TRUE,
            privilege    => 'connect');

        EXCEPTION
            WHEN acl_already_exists THEN
                DBMS_NETWORK_ACL_ADMIN.ADD_PRIVILEGE (
                acl         => 'sendmail.xml',
                principal   => 'SCOTT',
                is_grant    => TRUE,
                privilege   => 'connect');
        END;

        DBMS_NETWORK_ACL_ADMIN.ASSIGN_ACL (
        acl         => 'sendmail.xml',
        host        => 'mail.acme.com',
        lower_port  => 25,
        upper_port  => 25);
    END;
    /



    COMMIT;

    COL acl FORMAT A25
    COL host FORMAT A15
    COL principal FORMAT A16
    COL is_grant FORMAT A8
    COL privilege FORMAT A12
    SET LINES 120

    SELECT acl,host,lower_port ,upper_port FROM DBA_NETWORK_ACLS;
    SELECT acl,principal,privilege,is_grant,invert

    FROM DBA_NETWORK_ACL_PRIVILEGES;


    Revoking privileges would be just:
     

    BEGIN
       DBMS_NETWORK_ACL_ADMIN.DROP_ACL (
          acl => 'sendmail.xml'
      );
    END;
    /
    COMMIT;

    Views

    SELECT * FROM dba_network_acls;
    SELECT * FROM DBA_NETWORK_ACL_PRIVILEGES;

     

    Further Readings

    1. Oracle® Database Security Guide 11g Release 2 (11.2)Managing Fine-Grained Access in PL/SQL Packages and Types 
    2. Oracle Base Article on this subject.

    Monday, July 1, 2013

    OEM Reconfiguration


    Problem 

    Database is registered with a new listener (LISTENER2) working at port 1522 instead of 1521. The original listener remains intacted but the database is not registered there anymore.

    Result 

    OEM does not work anymore

    Fix

    It looks like following step should be enough:

    emca -reconfig ports -PORT 1522

    However the updated configuration does not work for the new listener name, the following step helps:

    emca -deconfig dbcontrol db -repos drop
    emca -config dbcontrol db -repos create

    or
    emca -deconfig dbcontrol db
    emca -config dbcontrol db -repos recreate


    Useful

    A HTTP port where OEM is listening, set ORACLE_SID then run a following command:
    [oracle@if1 ~]$ emctl status dbconsole
    Oracle Enterprise Manager 11g Database Control Release 11.2.0.3.0
    Copyright (c) 1996, 2011 Oracle Corporation.  All rights reserved.
    https://if1:5501/em/console/aboutApplication
    Oracle Enterprise Manager 11g is running.


    Literature

    1. Oracle® Database Installation Guide 10g Release 2 (10.2) for Linux x86 - Appendinx E. Managing Oracle Database Port Numbers, Section E.5 Changing Oracle Enterprise Manager Database Console Ports.
    2. Oracle® Database Administrator's Guide 11g Release 2 (11.2) -
      EMCA Troubleshooting Tips