Prior to ASE 15.7 SP100, ASE's errorlog had default permission settings of  rw-r--r-- (644).
Starting with ASE 15.7 SP100, SAP has applied a stricter security standard of rw-r----- (640).

This has caused problems with some third party tools that monitor the errorlog but have not been

set up to be part of the sybase group.

The OS UMASK setting can be used to enforce a more restrictive setting, but not a less restrictive one.

There are no configuration parameters you can change in ASE to make these permissions settings.

However, there is a workaround.  Less restrictive permission settings can be accomplished by having the
RUN_SERVER file execute in background a script that does  brief sleep and then a chmodon
the errorlog to the desired permissions.  This script execution would be put before the call to dataserver.
The sleep is to give dataserver enough time to have started up and applied SAP's restrictive permission settings to the
errorlog.

Create a script file named "reset_errorlog_perms" with the following contents (substitute actual  desired permission setting and errorlog name)

----------------
sleep 5
chmod 644 errorlog.log
-----------------

Set execute permissions on for the file (chmod 700 reset_errorlog_perms)

Modify the RUN_SERVER file so it calls reset_errorlog_perms in the background before calling dataserver.

SAP Adaptive Server Enterprise (ASE) has been a certified data repository for SAP Business Suite for some time now.  You may not know it, but SAP ASE is also certified with SAP BW as the main store as well as with SAP IQ as the Near-Line-Store (NLS) to BW.   You can find the report here if you have a support account:

SAP certifies Business Warehouse with SAP ASE database to support SAP IQ ODBC near-line storage

SAP BW running on ASE with IQ as the NLS allows lower TCO by optimizing storage costs.  Hot Data, that is information that is read and/or written frequently, is best kept in ASE on faster storage.  Warm data, that is information that is infrequently accessed can be kept in ASE on slower, less expensive storage.  Cold data, information that is rarely access and never changed, can reside in the IQ NLS in a compressed format on less expensive storage, yet still immediately available to BW if needed.  All of this is transparent to the end user.  You can learn more about the NLS solution by going here:

SAP BW Near-line Storage Solution (NLS) Based on SAP IQ

Conceptual Architectural Diagram:

At the end of January, SAP made an announcement that quite simply is revolutionary for any data architect. The announcement was the release of Adaptive Server Platform Edition (ASPE). Is this of interest to anyone not currently deploying ASE? Absolutely, yes. In fact, I’d advise database users of all flavours to read on. For ASPE is a data architect’s dream.

ASPE is a new licensing model that entitles the customer to perpetual license deployment rights to a core of their choice of ASE, Replication Server or IQ (as I outline below, ASPE also includes some important previously licensed options at no additional cost). With ASPE, users can switch between the cores whenever required, giving deployment flexibility to change their data management landscape to match business requirements. As long as capacity exists within the ASPE license estate, no further licenses are required.

Significant benefits

The possible benefits for user organisations are significant, not only in terms of business responsiveness but also cost. ASPE will, without doubt, lead to lower Total Cost of Ownership (TCO) across organisations’ license estates. With careful planning, the potential to decrease license costs for current and future projects by leveraging the flexibility that ASPE provides is considerable. Added to this, SAP is allowing SAP customers to trade in their existing under utilised licenses in exchange for new ASPE licenses, creating yet another opportunity for cost reduction. And finally, as if these first two points don’t provide sufficient incentive to consider ASPE, it also includes many options that were previously additionally licensed, presenting immediate cost saving opportunities as well a solution to meeting business, regulatory and security needs.

The additional standard functionality within ASPE is certainly enticing. ASE Encryption, ASE Semantic Partitions, ASE Security and Directory Services, ASE Compression, IQ Advanced Security and IQ Very Large Database Management are extremely useful tools in their own right. Below I’ve included links that provide extensive information about each option but, from a user viewpoint, the potential applications are extensive – here is a selection:

• Implement ASE data compression to cut storage costs
• Implement semantic partitioning in ASE to increase performance, reduce maintenance times and improve availability on very large data sets
• Utilise ASE encryption to address regularity and business security requirements with no changes to application code – including removing access to business data from administration personnel
• Deliver granular level access control to data with ASE Security
• Extend security with encryption of data between applications and databases with ASPE security
• As data volumes and analytical requirements grow, migrate business intelligence and reporting functions to IQ, delivering industry leading query performance combined with a reduced storage footprint
• Utilise IQ to meet corporate governance and regulatory requirements for information management
• Utilise Replication Server to keep IQ synchronised in near real time from your OLTP systems
• Extend data security out to your IQ systems with data and network encryption

ASPE Replication Server Platform Edition also provides replication services for SAP databases and Hadoop, in one product. This can be extended to replication to and from Microsoft SQL Server, IBM DB2 and Oracle with a single licensed option.

A fantastic package

This level of flexibility in application licensing is unprecedented and to my mind, shows that SAP listens to its customers, providing them with the greater technological agility they need in the modern business landscape. The combination of this agility with the potential for lower TCO and additional functionality makes ASPE a very compelling and highly usable tool that I will be recommending to my customers.

Chris Jones, J2 Consulting

Useful links

ASE Encryption– Protect sensitive data with minimal performance impact

ASE Semantic Partitions– Partition data to deliver lower cost and higher performance.

ASE Security and Directory Services– Ensure data privacy with row level access control, encryption of data in transit and integration with Active Directory, LDAP and PAM module.

ASE Compression– Compression of structured and unstructured data, reducing physical storage costs; improving cache and I/O efficiency.

IQ Advanced Security – Ensure the security of your data with encryption, authentication, authorization, and auditing.

IQ Very Large Database Management - Deliver simple, effective Information Lifecycle Management.

As many of you well know, back in the old, old, old days (when my hair was brown), ASE only offered allpages locking (APL).   In the move to data only locking (DOL) nearly 20 years ago (yes, it nearly has been that long!!), we added the concept of a latch.   One the hard parts of engineering's job is that they just don't get to 'implement row level locking' - they have to do extensive testing about where bottlenecks are and define alternatives as part of the implementation to come up with the best solution.   In the case of APL locking, it was noted that most lock contention and deadlocks were actually due to index pages - especially any index (and we all have them) on monotonic sequences - such as TradeDate - in which we were all fighting over that last page.

At the time, the solution was to use a non-transactional latch.  Whereas locks were held for the duration of the transaction and consequently a slow insert (or a large one) could keep readers waiting ....foreeeeveeerrr...., a latch would only be held long enough to make the change on the index page.   You all knew this, of course - I mean, we only beat this to death a slew of sessions in 1998 or soooo......

What you may not have thought about is that we had both "shared" latches and "exclusive" latches - although it makes sense.  When someone is reading an index page, we can't have some writer arbitrary whacking data in the middle or who knows what our poor reader would see.   And since readers aren't modifying data - yes...we allow concurrent readers to share latches.   However, when modifying the index page, the writer grabbed an exclusive latch.

Now, some of you may also have noticed that we are talking about "pages".....but wait - if we had datarows locking, why didn't we simply lock the index rows being modified.   The answer is actually quite simple.   Remember, most indexes are not unique.   For non-unique indexes, most DBMS's (MSSQL, Oracle, etc.) simply store the key values once and then have a RID array of values for all the rows that match the index keys.   In a shameless plug for my session at ISUG TECH (ISUG-TECH Conference, 2015 - March 29 - April 2), I am giving a session on data structures that discusses such basics before it gets into the real fun of compression and encryption.   Needless to say, the concept of an "index row" isn't quite the same - hence we work at the page level.

Fast forward a few decades.   Now transaction volumes have gotten much higher by orders of magnitude we can only blame on decimalization, web-ification, flash mobs on your website or whatever man made disaster..errr opportunity afflicting your business volume.   Luckily, Intel, IBM and other HW have been boosting CPU speeds at the same order of magnitude.   ASE is now capable of executing 2.5 million transactions per minute using an internal version of a standard industry benchmark running on commodity hardware.   And it is only going to get better - faster with sp02.  

However, that neat little trick of non-transactional latches......wellllll....now is starting to resemble the same problem as we had with index locks.   We are now seeing a lot of latch contention between writers and readers/writers.

Enter the ASE Engineering Super Hero team <cue sound track>.    No seriously, ASE engineering came up with a really neat solution for ASE 16sp02 called "Latch Free B-Tree's" (my boss is betting that I will get tongue tied when discussing this and Latchless Buffer Manager at ISUG TECH....but that topic is a good one for another blog).   In the Latch Free B-Tree, readers don't use latches.  Neither do writers.  Instead, writers use an in-memory map (you just knew I was going to say "in-memory" something, didn't you??)  and a delta/merge process to avoid latches as well.   And since they are writing to a delta area, there is no need for the readers to have a latch.   Keep in mind that we are not just avoiding the latch contention with this...we are avoiding the entire process of getting the latch at all.  Zero.  Nada.  Zilch.   If you remember from locking, we have lock hashtables, spinlocks and other nasty things that adds to lock overhead - while latches aren't nearly as bad....we still have to keep a list of them around somewhere, so there is some overhead that now...just....doesn't exist any more.   Gone.  Poof.

To say that this will improve transaction processing speeds would be like saying stepping on the gas pedal makes the car go faster.....     assuming you have it in gear, of course....and not in park.

Come to Atlanta if you want to hear more about ASE 16sp02 or keep your eyes peeled for webcast announcements to come.

Throughout 2015 too, there have been continuous request for the new license for the tools I have posted here in the past.   I am sorry for not being able to repack all the collections available here with the new license at this time as well as roll up the version updates for the tools.

All the tools are collected here:  Location for the Sybase Monitoring Tools Has Moved&amp;#8230; | Andrew m.

I have uploaded the new license applicable for all of the tools posted on my blog (available here:  SAP Sybase Tools: License Update (2015) | Andrew m.). 

You may also download the new license directly from here.

I hope you enjoy using these tools.  From my understanding of where SAP takes the DB product (and its maintenance/management) the only tool SAP will cater for DBAs is DBA Cockpit (for those lucky to have SAP application packages) and SAP central (for those who do not).  DBA Cockpit is expected to have even load replay functionality at some point...  So, talk to your SAP reps...

Enjoy the tools and have healthy and productive life.

ATM.

ps.  see you at ISUG 2016 in Atlanta...

ASE 16.0 SP01 is now available for download from http://support.sap.com

Cheers,

-bret

Sybase has been in SAP hands for almost 5 years now.  What did it mean to Sybase DBAs?  Did we (if I may reference myself) experience any difference?  Do we feel better or worse?  Did it make any difference at all for Sybase DBA community in general?

In my opinion, 5 years is a sufficient period of time to draw a line of now & then.  Some things did change.  Some trends have crystallized.  We changed too.  Or did we?

One of the major difference for non-SAP Sybase DBA community has been the demise of most of the old sites from Sybase.  Most of them now reference to maintenance.sybase.com bluntly.  Which is fine, as it is.  What is less so is that if once it has been very easy to locate Sybase-related technology information on the web (e.g., go to http://www.sybase.com and continue from there...) today it is quite a daunting task.

Say you go to www.sap.com and try to look up Sybase DB technology there.  Where should you look for it?  Technology perhaps?  So you go to solution->technology (http://go.sap.com/solution/platform-technology.html).  Found it?...  Nope.  Not there.  The easiest way is to search for it using the site search interface.  Trying to locate it "logically" is frustrating.

Sybase database technology information is now accessible from help.sap.com portal (http://help.sap.com/database or, for ASE, http://help.sap.com/adaptive-server-enterprise/).  The old good friend of Sybase DBAs (http://infocenter.sybase.com) is now under the rubric of "Non-Supported Documentation."  All the PDF versions of the new documentation are available from help.sap.com, but the search engine of infocenter.sybase.com has been so much more convenient.   There is no longer free of registration search engine available as of old.  You do have the new one in place (https://support.sap.com/home.html), but it requires registration with SAP first.  You are also encouraged to use SAP community network (http://scn.sap.com/community/ase-custom-applications or http://scn.sap.com/community/ase).

Which brings me to SAP marketing policy on Sybase.  SAP's umbrella has been definitely beneficial to Sybase ASE product.  New features are being poured into it.  Extreme performance is targeted.  Simplicity too.  But this comes at a price.  In a sharp contrast to either Microsoft or Oracle, SAP tends to keep information on its database product available mostly ... to it own community.  If you happen to have access to SAP marketplace you are open to a lot of information on Sybase products.  If you are not - you are in a disadvantage as far as your non-Sybase colleagues are concerned.  One thing that did not happen in an aftermath of SAP purchase, much to my disappointment, is that Sybase reached the masses.  SAP is everywhere (all of the clients I have attended as a consultant have a SAP product on the IT shelf).  Sybase is nowhere (most of the clients I have attended haven't heard about it, and those who do work with ASE as a platform for SAP applications have no idea how this "free" DB works and usually see it as an underdeveloped database version compared to DB2/Oracle database technologies they were used to work with).

This brings me to the Sybase DBA vs SAP DBA Cockpit issue.  Every Sybase DBA knows that as far as Sybase is concerned you'd better write tools on your own than wait for some vendor to get interested in Sybase technology and come out with a good tool to manage it.  This too did not really change.  I've seen large vendors dropping support for Sybase (Dell e.g.?) but haven't seen a reverse trend yet.  Given SAP marketing policy of keeping a product to itself it will hardly change in the near future.  Here too Sybase DBA and SAP DBA fare differently (???).  SAP DBA doesn't have to worry about its DB.  It is (now) a-thing-in-itself.  SAP envisions ASE DB as a DB solution that takes care of itself.  All you need is your DBA Cockpit to be installed and configured properly.  The rest is history.

If you have your ASE DB incorporated into DBA Cockpit it will manage statistics automatically, treat fragmentation on its own and auto-expand when needed.  It will even allow you to walk through MDA statistics collected into 30 or so nice blue windows (Waits, Spinlocks, Table Statistics, & many more).  So SAP DBA will probably have no idea how the DB supporting its SAP application works, but he/she will have little need for that since Solution Manager will take care of the DB for him/her.  And if the need will arise to peep into how the DB actually fares, he/she will have rudimentary MDA metrics collected and archived for his/her convenience.   Sybase DBA, on another hand, will have to continue to struggle on his/her own.

So did or did not the SAP purchase of Sybase make any difference at all for Sybase DBA community in general?  It definitely did.

In my opinion the purchase has introduced a rift into it.  There are now Sybase (SAP) DBAs and SAP (Sybase) DBA Cockpit operators.  Instead of empowering the former, SAP marketing (?) policy has given rise to a sub-community with less knowledge but more privileges which has not been really meant to dive into DB technology in the first place (see, for example, how many sessions on DB technologies are there in SAP TechED && d-code events).  To me it is pretty unfortunate.  In order to transform ASE (&c) into the second (or even third) major DB technology one has to bring communities together and open up technology to the wider public rather than introducing differentiating factors into that community and aiming to control access to technical information.  It is not enough to make the technology better.  It must be better known, better presented and must make the lives of those used to take care of it daily for years more comfortable.

As far as I am concerned I still wait to see a change in SAP agenda.  DB technology is not just another product in SAP PAM.  It is a technology.  And it has its community out there.

ATM.

SAP issued the SP01 release for ASE 16 a couple of days ago. 

Having done so, SAP continues with its stable pace of release-a-quarter for its ASE product:

SP01, however, is a major release - at least based on its content.

If we have a close look at the bug fixes incorporated into this release we may indeed be impressed.  Talking in rough numbers, ASE has a general rate of about 600 bug fixes a year.  Given the quarterly release policy we might have expected to see around 150 fixes in SP01.  However, SP01 has come out with... 536 unique bug fixes since SP0 PL5.  This is a huge number.  I'd definitely put an effort to check this release with the real application load and see how it performs.  Worth an effort.

From the functionality point of view, the release isn't as fat.  It has not introduced any new MDA metrics since the SP0 PL5 (the same 1604 distinct counters).  SAP has squeezed all the new features that are there into one neat page in documentation (as opposed to 12 pages on 16.x release in general).

These are the features added:

I wont copy the digest from the documentation here.  You may simply read it on your own on the official SAP help site (http://help.sap.com/ase1601/#section2).

From the SCN posts we also know that SAP engineers work on unleashing more transactional power for ASE (by rethinking its latching mechanism).  ASE 15.7 has been able to boost TPS power twice (or even three times) as compared to ASE 12.5.4 based on my personal experience with some of its clients.  I am eager to see the TPS rate for 16.0.1.0 (or the release which will incorporate the new latching mechanism - who knows, may be it is kept secret for ISUG 2015...).  Especially since now IMDB OLTP instances are being set up by the real production ASE systems.

ASE is here to stay.  If only its marketing has been more visible.

But that is another story.

Have fun with the new release.

Cheers,

ATM.

Finally the tickets are in my pocket and I may start arranging the suitcase...

Hope to see you all at ISUG 2015 conference to take place in Atlanta!

See the details here:   ISUG-TECH : ISUG-TECH Conference, 2015 : Content Landing Page

For all those interested in ASE (and other ex-Sybase) technology this the place to be...

Cheers all,

ATM

ps.  Feel free to attend my session ...

Depends on whom you ask, I am not often wrong.   This time I was ....and foolishly put my $$$ where my mouth was.   At ISUG this year, I was presented several sessions - one of which focused on data structures and segued into data compression and data encryption - with the idea of showing why compression and encryption could have the impacts that they do and when to best use (or not use them).   A bit of a heady topic to jam into 2 hours.   However, during my development of the materials, I decided to take on the challenge that my friend Cory Sane had issued last year at the enhancements panel:  why doesn't expected row size behave the way we would like it to?

For those of you who have no idea of expected row size, shame!!!   You really must like running reorgs.....and exclusive table locks.

Back nearly 20 years ago when ASE added datarows locking, a number of features were added to alleviate common locking contention points.   For example, the concept of using non-transactional latches vs. locks for indexes.   One of those features was the notion of an "expected row size".   The problem with row level locking was that if you updated the row and the row expanded, often we were forced to forward the row to a new page as the new size impinged on the next row on the page.   While we could have simply locked the rest of the rows on the page, for common situations such as heap tables, this would lead us right smack back into the problem we were trying to solve - hence row forwarding.   To prevent row forwarding as much as possible, we added a table attribute "expected row size" that can be set via "create table" or via sp_chgattribute.   When set, this preserves enough space on the page so that full rows could be written....but didn't reserve the space within the row.   As a result, there were some situations in which some row forwarding still occurs.   Cory's challenge was we should fix it.   Engineering's response was that it would be extremely difficult to near-impossible......

So, partly due to curiosity and partly to answer the challenge, I decided to figure out why.   The answer was in the row format.   To make the expected row size attribute work as desired, we would have to add another 2 bytes to the row format - which presents a nasty issue during upgrades as this would involve re-writing every data page with datarows locking....and due to the row expansion, likely would involve page overflows/new allocations - which would in turn lead to index RID updates, etc.   Ugh.  No one wants an upgrade that takes 30 days.....

In the session, I did point out that it is clearly documented that if you have forwarded rows and run reorg forwarded_rows - or any other reorg that purportedly works in "small transactions" to avoid contention, that as soon as the reorg hit a forwarded row, it escalated to an exclusive table lock for the remainder of that "small transaction"....doesn't sound so bad....until you realize that those 3000 forwarded rows becomes 3000 exclusive table locks - just one of the many reasons that using exp_row_size to dramatically reduce forwarded rows is much better than running reorgs to begin with....or at least upgrade to ASE 15.7 sp100+ and use one of the online reorgs.

However, as part of the discussion, I demonstrated how expected row size was multiple orders of magnitude better than tables without the attribute set - showing that for my example, it reduced row forwarding from nearly every row to just low thousands (or high hundreds) instead of 100,000.   I also pointed out that this is where I was typically lazy - rather than trying to compute the row size as documented in the manuals (x bytes of overhead, plus x bytes for each variable length column, etc.), I simply tried to set the exp_row_size attribute to something stupidly large (e.g. 4000 bytes) and ASE would whine at me about the size being larger than the maximum of ### bytes.....and so I would then simply reset it to the maximum as computed by ASE.

At that point, things went a bit tricky.   Kevin Sherlock challenged me with whether this was the logical or physical rowsize being returned in the error from ASE.  I had assumed the physical (to include all row/column overhead) vs. the logical (just the datatype lengths - no row overhead) and said so.   He pointed out that if it was the logical vs. the physical, it might explain the little row forwarding that I ended up with......meh.....maybe.....maybe not (actually, not as we still have to deal with contiguous vs. non-contiguous free space and the role of the housekeeper - which we didn't get into).....and knowing that, I challenged him right back.   Bad move....Jeffy....bad move.    He accepted with the loser paying for lunch sometime as the penalty (we are big spenders at ISUG).   The problem is that this is so easily disproven - even in my example, it was obvious that the answer was logical - I just wasn't paying attention.

Of course, now I have to figure out if engineering takes the 'logical' row size and computes a physical one that is used internally......I have an idea how I can prove that....and if so....then maybe he will owe me lunch after all.

...next time I am in Omaha...one of us will be eating steak and the other crow.....I just hope it tastes like chicken.

Now, about that "magic go faster" button.   During the keynote, it was my pleasure to demonstrate the combined power of SNAP (Simplified Native Access Plans - aka "compiled queries"); Latch-free B-trees (see my blog at http://scn.sap.com/community/ase-custom-applications/blog/2015/03/13/latch-free-b-trees-coming-soon-to-a-server-near-you for explanation); and Lock-less Buffer Manager.    To set the stage, the SPEED team and QA in engineering had set up two identically configured HW boxes - both with 80 cores and 512GB of memory.   On one, we were running ASE 16 sp01 - hot off the presses with all the advantages for contention in 16GA over 15.7....where most of you are still running.   On the other, we were running the latest engineering build of ASE 16 sp02.    Both were configured identically with 80 engines and gobs of memory - and identical data cache configurations, etc.   The only difference, of course was that the ASE 16 sp02 system had all three of the XOLTP features we were discussing enabled.   The benchmark we ran was an internal one from one of our internal systems used in real life every day (our case management system) - we simply scaled it from 1000 to 5000 users and plotted the results in both response time and throughput.    For those of you who wimped out on ISUG and didn't get to see the demo, here is a screen snapshot of the results from one of the many practice runs:

Okay, so it might be a tad small....but it shows that sp02 had 7x( !!! ...that's 700%) better throughput and 90x improved response times!!!   Whoa!!!!   When is the last time you saw anything greater than 2-3x - without any code changes?????   Of course, the issue is contention - which isn't always consistent - for example, one of the other practice runs had a slightly different result with:

Still - nearly 4x better with sp02 vs. sp01.    I wanted to show this last chart for a reason.   Contention is often a matter of timing - and as beautifully illustrated by the red sp01 line in the bottom chart (response time), contention can be rather imprecise and unpredictable - sometimes worse than others - which is why it is such a nuisance as you may think you have fixed it...and presto - it comes right back at you.

How realistic was the test???   Very - I have frequently observed high cache contention in a number of customer cases - especially in FSI customers where one of the most common queries is for recent stock pricing.   Last June for example, I was proving to one FSI customer in London that the threaded kernel was indeed faster than the process kernel - and in their case what we quickly learned was that the single biggest scalability problem in their application was cache contention.   I could only say "sorry.....but ASE 16 will fix that"....and it does as shown above.   Even without user data entering into the picture, for one customer, we determined that the biggest impediment to scaling the application was cache contention on sysusers - even though bound to a separate named cache (just that one system table)...all due to the fact the app connected via a single login (even if it used several, due to page size, it still would have been an issue).   For those of you who insist on not listening to my rants about using a dedicated named cache for system tables (and indexes) and still have system tables in default data cache....welllll.....let's just say that if you really want a fast way to reduce your default data cache spinlock contention, create that named cache as I have said many times over.....

....and then in sp02, tweak it by enabling the lockless buffer manager to eliminate all cache contention.

In the next months, myself and my colleagues will be taking to this space to bring you more details about the Corona features in sp02.   Topics will include things such as Transactional Memory, SNAP, lockless buffer manager, HADR, capture replay (I got to demo that too.....definitely cool!!)......so stay tuned.

....that would be next Tuesday for those like myself who don't have a built-in calendar function in their cranial cerebrum.....

Now that tax day is over (for those of us here in the USA.....my friends in Canada still got a few weeks of fretting to do), we can get on with life - and that means keeping up with where technology is going as well as doing our jobs.   I put it that way as often we don't have time while doing our jobs to keep up with things, do we???

Anyhow, next week, I am presenting in a webcast about ASE Cluster Edition (ASE CE) - both from what is available today as well as where we think we are going.   If you got the email about it, you probably saw that topics for discussion include:

• A brief/quick history of ASE clustering (for those who are not familiar with all of our past successes....and not so successes)
• A brief walkthrough of ASE CE workload management and how this applies to availability
• A discussion on SP130 features including Single Instance Database (SIDB) and cluster locking enhancements
• A fairly thorough discussion on ASE CE and rolling upgrades (imminently available)
• ASE CE 16 roadmap

If you missed the email or it went SPAM folder automatically, you can register here:

Event Registration (EVENT: 963386 - SESSION: 1)

With respect to the discussion topics, the last bullet point is one that I think we need to be a bit more upfront about.   We have often discussed roadmaps in a general sense and maybe in some cases referred to some specifics with respect to planned functionality.  However, as an ex-DBA myself, I know that it is often desirable to understand some of the challenges and decisions that may drive product development one way or another....vs. waiting until the product is GA and then trying to find out *why* some engineer made a particular decision that seems sooooo... counter-intuitive.   I have made it part of my goals in product management to help customers understand why certain implementations were chosen - as much as a way of educating about the technology but also simply to head-off a lot of the "why didn't you ...." objections/questions raised a lot around key topics.  As a result, we will have a short discussion during the webcast about metro/stretch clusters (or whatever your preferred term is) so that we all understand some of the challenges we are facing in some of the future roadmap items.   This should be a key clue that if such features are important - now is the time to help us understand your requirements so as to help us set priorities.

We also will have a short discussion on horizontal scalability - although there are some internal folks at SAP that might be a bit concerned about that.  To me, not discussing it is like ignoring the enraged bull elephant in the same room as you are....and it's between you and the only door you can escape out of.  Oft times, I have found that those really desiring horizontal scalability don't really realize that shared disk clusters have inherent limitations in this area.   In fact, those pursuing horizontal scalability usually break out into one of several different classifications:

• Those looking to exploit the cluster hardware vs. a sunk cost (TCO play) by spreading workload across the machines
• Those looking to recapture some of the performance overhead of SDC on their applications be leveraging the other node to make up for any loss of performance
• Those that actually could leverage such a solution (rare - but exists) - but may not understand how SDC implementations may frustrate their best efforts
• Those who have read about it and in their irrational exuberance embark on a science project...  incontrovertibly destined for failure

....so, we will take a brief look at why applications think they need horizontal scalability as well as what technologies are actually necessary to achieve the scalability or to work-around issues.   Don't get me wrong - clustering overall is a common technique for scalability - Hadoop/Map Reduce and many other implementations thrive on it.   But shared disk clusters, such as ASE CE, Oracle RAC and IBM DB2 PureScale - all share a lot of similarities with scalability limitations based on what we used to call computer science 101 issues.....or perhaps it should be 301 as it definitely is a bit more of a complex topic that programming "hello world".

See you on Tuesday!

Most of us are paranoid about security – be it in our work lives or our personal lives.  Having said that, 2014 Verizon Data Breach Investigations Report states that with all the regulations in place, in 2013 alone there were 63,437 (an increase from 47,000) security related incidents of which 1,367 (an increase from 621) of them were confirmed to be data breaches.  Most of these incidents were opportunistic attacks which means that they could have been prevented.  There is a high requirement for tighter internal controls to prevent unauthorized disclosure of sensitive information.  How are these being addressed from a database perspective?  SAP ASE’s encryption and security technologies protect data from both internal and external security breaches allowing you to keep your data secure.

About 10+ years ago, ASE introduced the state-of-the-art column-level encryption which enabled administrators to protect sensitive data on disk and help meet security and privacy requirements.  And with SAP ASE 16, ASE added support for full-database encryption.  Both encrypted columns as well as fully encrypted databases can help companies meet their security and privacy requirements.  One solution however, might be more suitable than the other, depending on specific circumstances. Column-based encryption might be preferable when a company can easily identify what data is sensitive for their business and different database users have different access rights on the sensitive data.  Column-level encryption requires that each sensitive data attribute be evaluated individually to validate the access rights of different users and to identify the encryption technique used on the attribute.  Even though it has its own advantages in terms of granularity, it has its dis-advantages as well.    In specific scenarios, performance can be a concern.  One example is when queries use common techniques such as range searches on the encrypted data field.  This is because the encryption process will likely randomize the data such that it is no longer in alphabetical sequence.  Another example is during data retrieval.  This is because data is not only kept encrypted on disk, but also in main memory.  So in this case, for each user accessing the data in the encrypted column, decrypting has to be done.  Also one more limitation is you cannot create an index or have a constraint on an encrypted column.

On the other hand, full-database encryption may be beneficial for companies that have different requirements.  The use of full database encryption is preferable when it is hard to identify which columns hold sensitive data.  With full database encryption, we do not have to worry about identifying columns that have sensitive data.  This is helpful especially since the definition of sensitive data differs from one location to another.  Also, it might perform better especially if all database users have the same access rights (or a single database login is used). The fact remains that full database encryption is transparent to the user and the application as there is no change to the way one would query the data or manage the data.    In SAP ASE 16 full-database encryption automatically encrypts all database objects – including data files, indexes and transaction logs.  The entire encryption process is transparent to users and existing applications because the data is automatically decrypted when it is moved from disk to memory, and encrypted when it is moved from memory to disk. Encryption uses 256-bit symmetric keys and is implemented at the page level.  Buffer manager is used to encrypt pages before they are written to disk and decrypt them when they are loaded into memory.  One aspect of this functionality that is really very helpful is that it is implemented as an online operation. This means that an alter database command that fully encrypts a database executes in parallel with other database workloads and doesn’t interrupt any activities – be it – executing  queries or doing an insert and update operations.  Let us say we are going to encrypt a very large database.  We know that it can take a long time.  The good thing is that the encryption process can be executed along with other activities or it can be suspended and resumed, giving DBAs the flexibility to perform critical system maintenance as needed.  If you remember, we had done this kind of mechanism with our maintenance activities like reorg operations and also creating an index giving more flexibility to users.

In closing, deploying technology that reliably keeps your enterprise data secure has become a necessary best practice to comply with security regulations.  And SAP ASE does it once again with adhering to security using encryption – both column level encryption and full database encryption.  Encryption algorithm in ASE uses FIPS140-2 certified modules for implementation.

I should have written something down about this 3 years ago.   That's when I made a comment to a friend at the ISUG Tech conference that expected_row_size isn't exactly what you might think it is.  In fact, even if you set it to it's maximum allowable size, you can still see forwarded rows happen.  And, that's really what the docs say.  Well, it's subtle anyway.    Then, at this past year's conference (2015), I see it mentioned in a couple of presentations.  That prompted me to go find my old research, and try to dig a little deeper.

As a result, there is too much information to really make effective use of a "blog" entry, so I decided to create a series of videos on the subject.

The first video

SAP ASE 16 - What to Expect from Expected Row Size - Part 1 - YouTube

just discusses the topic and it's current documentation.

The second video

SAP ASE 16 - What to Expect from Expected Row Size - Part 2 - YouTube

goes into a demonstration and case study of the various behaviors, metrics, and how transactions can affect what rows get forwarded, and what rows stay on the page.

Space management is an interesting (and complicated) topic.  I'm sure those here who have access to the ASE code can elaborate more on the topic, but these are simply my observations from "the outside".  Comments are welcomed!

Introducing ASE Cockpit

ASE Cockpit is planned to be a new User Interface (or Tooling) component for ASE, and is planned to be released in 2015.

The User Interface (or Tooling) of ASE Cockpit will manage ASE. It is targeted as an essential companion tool for a Database Administrator (DBA) of ASE. In comparative terms, ASE Cockpit release in 2015 will be to ASE as:

1. HANA Cockpit released in Q4 2014 is to HANA
2. IQ Cockpit released in Q2 2014 is to IQ
3. ESP Cockpit released in Q2 2014 is to ESP
   

The "Cockpit" approach is being adopted and launched by several SAP Database products as a concerted approach to simplified and unified tooling. There are several underlying characteristics of these Cockpits, some of which we will address below. (All possible angles of this vast topic have not been covered in this blog. But a few more blogs are planned in future, that may eventually do justice to this topic.)

Is ASE Cockpit somehow related to DBA Cockpit?

No, ASE Cockpit is different from DBA Cockpit. They share the same suffix "Cockpit", but their scope (or roles) are different.

DBA Cockpit (and Solution Manager) are distributed with SAP Business Suite (or SAP Netweaver stack) products, and not distributed with ASE standalone database.

Furthermore, DBA Cockpit is meant to be used by a System Administrator (SA) to manage the underlying databases in the context of the SAP Business Applications that are using that database, and not necessarily the detailed database management that is typically required of a DBA. ASE Cockpit, on the other hand, is intended and targeted for the tasks and roles of the DBAs of SAP ASE.

To summarize, both ASE Cockpit and DBA Cockpit have their role to play, have some functionality overlaps, and are planned to continue. They each will still focus, position and align on their unique scope for their intended role or persona.

One ASE Cockpit for One ASE Server (1:1)

The term "Cockpit" signifies a "console" that can directly operate the device or server that it was installed with. This implies, there is a 1:1 association of an ASE Cockpit with its ASE Server. This is just like the 1:1 association of an Airplane "Cockpit" with its aircraft. So, the ASE Cockpit to ASE Server relationship across several instances of ASE Servers will be as follows:

1. ASE Cockpit #1 will manage ASE Server instance #1
2. ASE Cockpit #2 will manage ASE Server instance #2
3. ASE Cockpit #3 will manage ASE Server instance #3
4. so on ...

This is similar to how Cockpits for HANA and IQ operate with their deployed Servers as well. See the diagram below that shows ASE Cockpits 1:1 association alongside other SAP Databases that also have their Cockpits.

Figure: Illustration of Cockpits of ASE, HANA and IQ

The release vehicle for a Cockpit component will be their Server product. And the same is true of ASE Cockpit that will be released with ASE releases.

ASE Cockpit up-and-ready with minimal setup

Cockpits across SAP Database products are providing an experience of readiness; i.e., Cockpit gets installed by default with its Server product, and is up-and-running to manage the Server it was deployed with. The same is true of ASE Cockpit. As soon as the ASE Server is installed (or deployed), its ASE Cockpit is expected to be able to manage it with minimum additional steps.

Compare this expectation of simplicity in ASE Cockpit with that of SAP Control Center (SCC) 3 for ASE. SCC 3 requires a separate installation after ASE was installed, a separate start of the SCC 3 service, separate registration of ASE Servers, separate registration of Agents, assigning the ASE Servers to appropriate perspectives, etc. Many of these steps have either been eliminated or simplified in ASE Cockpit.

ASE Cockpit to have server-side logic and browser-based UI

For further clarity on platform and version compatibility of Cockpit instance; note that a Cockpit component is co-located with its deployed Server instance. This means a Cockpit supports the same machine, version and platform of Server with which it was installed or deployed. This is going to be true of ASE Cockpit as well; in that the ASE Cockpit is going to be co-located on the same machine+version+platform as its ASE Server instance. For comparison, SCC 3 for ASE was supported on fewer platforms than ASE, whereas the ASE Cockpit component is planned to support every platform its ASE release vehicle (or version) supports.

Cockpits' user interface (UI) are taking a browser-based UI approach across SAP Databases; and working towards eventually replacing any Eclipse or other thick-client desktop-application UI. ASE Cockpit user interface will also, similarly, be browser-based UI. There are some good justifications to having a browser-based UI for ASE Cockpit:

1. Firstly, the intent of making ASE Cockpit user interface to be browser-based was to allow remote access to its ASE Server, simultaneously from various users using a variety of user devices that support a browser. Even as the actual ASE Cockpit logic resides co-located on the ASE Server machine, browser-based UI access would enable the user to use his choice of device (with a browser) to manage the ASE. For instance, this would prove useful in scenarios where the ASE Server is located in a Data Center or Cloud, while the user manages it via a user device with a browser.
   
   
2. Secondly, a single "software patch/upgrade" to the server-side instance of ASE Cockpit component would update the ASE Cockpit UI of all users and on all their variety of devices, at once and consistently. To appreciate this, realize that such an "one-shot upgrade" of UI on all your devices saves you the effort of otherwise requiring to upgrade the same desktop-application on all your devices one-by-one.
   

What happens to SCC 3 for ASE going forward?

SAP's direction is to invest in Cockpit approach going forward for its Database products. For the ASE releases that provide ASE Cockpit, SCC 3 for ASE may not be packaged in that ASE release.

SCC 3 for ASE will still be supported for previous ASE releases that are in support. In a transition phase of introducing and launching ASE Cockpit, where ASE Cockpit adoption and migration from SCC 3 take place, SAP will monitor and review the needs of existing SCC 3 for ASE customers.

Thoughts?

Please call or write to us with your inputs and feedback on this plan/direction. A lot of these changes have come from the feedback we have received in the past, e.g., to simplify installation, setup and configuration for SCC 3 before you can get started to manage ASE with it.

We want to emphasize that the launch of the ASE Cockpit will just be the first step in our intended continuous process of innovations for ASE Tooling. And your inputs and requests would shape the requirements for our future tooling releases. Please contact us if you are interested in a Design Partner type engagement on ASE Tooling.

We understand and appreciate that each such transition or migration is not effortless, despite our best attempts and planning to make it simpler. Yet we believe we are heading down this path because we believe this will be the better, simpler, aligned, and unified plan across our SAP Database stack further down the path.

***

Coming next ... "What's new in ASE Cockpit?"

What's new in ASE Cockpit?

First, what is "ASE Cockpit"?

We introduced ASE Cockpit in the previous blog in this series, here: ASE Cockpit (in lieu of SCC 3 for ASE).

ASE Cockpit is planned to be a new User Interface (or Tooling) component for ASE, and is planned to be released in 2015. The User Interface (or Tooling) of ASE Cockpit will manage ASE. ASE Cockpit is targeted as an essential companion tool for a Database Administrator (DBA) of ASE.

So, what's new in "ASE Cockpit"?

ASE Cockpit will have all the monitoring and capabilities of SAP Control Center (SCC) 3 for its ASE, plus the support for the following new features/capabilities in upcoming ASE. Due to ASE Cockpit being a superset of SCC 3 in managing its ASE, ASE Cockpit will replace SCC 3 in the packaging of its upcoming ASE release.

1. ASE Cockpit supports more platforms than SCC 3 for ASE:
   • Added platforms: AIX and HPIA
2. ASE Cockpit supports new ASE features:
   1. Always-on Option (HA/DR)
   2. Workload Analyzer Option
   3. Data Store and Access Management (DSAM)
3. Other benefits of ASE Cockpit:
   1. Easier installation
   2. New look
   3. Simpler Alerting
   4. Localization
      

Now let's expand on each of the above new capabilities of ASE Cockpit.

+++

Increase in Platform support, and Easier installation

(1 & 3.i)

ASE Cockpit has to support more platforms than SCC 3 because ASE Cockpit is co-located with the ASE and has to support all platforms ASE supports. ASE platform support information is typically included in the product documentation. In contrast to ASE Cockpit, SCC 3 did not have to be located on the same host as the ASE.

But, why is this new capability or attribute of ASE Cockpit significant? This capability is significant not necessarily because a few more platforms were added, but because ASE Cockpit now co-locates with its deployed ASE. Co-locating the ASE Cockpit simplifies the installation and configuration needed before you can manage your ASE with it.

More importantly, this co-location and simplification of ASE Cockpit is important because this is expected to meet a key request/requirement from our ASE customers. As a quick flashback, the inputs and feedback we received from our ASE customers was to further simplify the SCC 3 installation. And this co-location of ASE Cockpit essentially simplifies the installation steps (vis-a-vis the SCC 3 installation steps).

The almost automated install, registering, startup, etc., by the co-locating of ASE Cockpit with ASE offers a significant progress in ease-of-use and simplification.

+++

Always-on Option (HA/DR)

(2.i)

New for the upcoming ASE release in 2015 is its "Always-on Option". The Always-on Option of ASE consists of components for High Availability (HA) as well as for Disaster Recovery (DR) implementation. True to its low TCO advantage and commitment of simplifying, with its release ASE will be combining HA as well as DR into one package.

Notably, the key point is that: ASE Cockpit will provide the User Interface (UI) coverage for the management of the ASE Always-on (HADR) environment.

Workload Analyzer Option

(2.ii)

This new option for the upcoming ASE release will enable Workload Capture and Replay of production workload non-disruptively, and utilize the data to diagnose problems, and to understand and manage configuration changes proactively. It will aid in:

• Identifying problematic queries
• Investigating client activities
• Measure impact from configuration changes
• Evaluate upgrade options and new features to understand their benefits
• Evaluate migration options by anticipating upgrade/migration benefits

ASE Cockpit will be the UI for this new ASE feature.

Data Store and Access Management (DSAM)

(2.iii)

The new DSAM new in the upcoming release of ASE can enable you to maximize the value of existing storage subsystem by taking into account the varying access frequencies over a period of time, the value of data during the complete data lifecycle, and savings through placing data on the most efficient devices. It will help in:

• Data usage monitoring
• Moving data to faster devices, or to archive devices, based on data access activity
• Enhance performance by keeping active data on high performance storage
• Reduce costs (enabling more effective use of storage assets)

The UI for DSAM feature will be in ASE Cockpit.

+++

The new look and feel

(3.i)

ASE Cockpit re-uses various components of its predecessor SCC 3, but with a refreshed UI template and a simplified top-level navigation.

ASE Cockpit will use a UI template consistent with the UI standards at SAP, and will make ASE Cockpit look similar to the IQ Cockpit and ESP Cockpit. (Note that ASE Cockpit will emulate the UI5 UX3 standard at SAP on its Adobe Flash UI, and not use Fiori HTML5 standards used by HANA Cockpit.)

In addition, ASE Cockpit will use the fuller real-estate of the browser compared to the windows-in-a-window layout of SCC 3.

And finally, the top-menu (Application and/or File menus) and top-level toolbar of SCC 3 are simplified with the following three tabs instead:

1. MONITOR - View the real-time and historical charts
2. EXPLORE - View database components and entities, and manage them through provided admin functions
3. ALERT - Manage the alerting setup, configurations and preferences
   

Simpler Alerting

(3.ii)

ASE Cockpit simplifies Alerting for its ASE to the ease of selecting a checkbox. Where SCC 3 required you to define/setup the name, preferences and thresholds of an Alert before you could apply it, ASE Cockpit simplifies this internally to set predefined thresholds that will trigger its alerts. This simplification gets you started on basic Alerting right after you install ASE. You can then modify these preset threshold values in the "Alert" tab.

ASE Cockpit uses Key Performance Areas (KPAs) as the metrics and/or aggregates for its alerts. These KPAs in ASE Cockpit are a replacement to the Key Performance Indicators (KPIs) in SCC 3. (In other words, the KPIs of SCC 3 have been reclassified in ASE Cockpit as KPAs.) These KPAs are classified into four categories: Availability, Performance, Capacity and Alerts (acronym "APCA"). ASE Cockpit internally has "Collection jobs" scheduled and run to collect these KPA statuses.

Localization support

(3.iii)

ASE Cockpit debut release will support localization in Japanese and Simplified Chinese. Note that, in contrast, SCC 3 did not support Localization. (ASE Cockpit, and SCC 3, are internationalized.)

***

Jump to a sub-topic in this blog:

***

Coming next ... "Which DBA Tool to use with ASE, and when?"

***

Appendix: Screenshots

(Click on a screenshot to see higher resolution)

• Login screens

• ASE Cockpit

• SCC3

• Look and feel

• ASE Cockpit

• SCC3

• Alert setup

• ASE Cockpit

• SCC3

Webcast: Accelerating SAP ASE Applications Using SAP HANA Don't miss our July 8 Webinar to discover how you can enable modern transactions and faster reporting

Register now

Running real-time analytics on transactional data is now a reality – thanks to the SAP HANA accelerator for SAP Adaptive Server Enterprise. Join us at our informative Webinar on Wednesday, July 8, at 12:00 p.m. ET to learn more about the SAP HANA accelerator for SAP ASE and find out how you can:  Run existing SAP ASE reporting applications faster on SAP HANA with no changes to code needed  Build new analytics applications using transactional data  Use real-time analytics to make intelligent transaction decisions

Register

now to join us and gain a more complete picture of your business.

For those considering to move up the ladders from their current ASE release (whether it is ASE 12.5.x or ASE 15.x) there are, obviously, two possible migration alternatives:  ASE 15.7 & ASE 16.   To make the decision more tricky, SAP has stamped both with the same supportability badge:  both have the same EOL dates somewhere in 2020.  Which should one choose, then, and what are the basic elements of making this decision?

Well, this depends on your upgrade methodology.

Binary overlay and its variants:

This one is the easiest to implement. Unload the binaries into the same base directory (or into a separate one if you choose a clean install - my favorite anyway). Run through pre-upgrade steps.  Run through post-upgrade steps. And voila.... Not really.  There is a fundamental difference for this method depending which target migration version you choose:

• For ASE 15.x based migrations the selection of the target version controls whether it is minor or major upgrade.  Staying on the 15.x base means there is a possible downgrade path ONLY if you choose to upgrade to 15.7 release.  So, if you are relying on replacing ASE binaries AND you are based on 15.x right now, jumping to ASE 16 crosses the no-return point. There is no downgrade path from ASE 16 to ASE 15 (or earlier).
• For ASE 12.x based migration the selection of the target version is of less importance from the downgrade perspective.  It is always one way.  There may be more intermediate steps, but the way back is the same.  None?

A thing to keep in mind for ASE 15.x based customers:  SP100 is a breaking point for ASE 15.x SPxxx targets.  There is no direct downgrade path from post SP100 ASEs to pre SP100 ASEs.  So if you do plan to choose ASE 15.7 for your target ASE release your downgrade will have to go through an intermediate stop:  first you will have to downgrade your new ASE to pre-SP100 release (relying on sp_downgrade_esd).  Only then you will be able to roll back all the way to your original release (relying on sp_downgrade).  This is a nuisance, but at least there is a possible downgrade path in case you hit some serious issues post-migrate (it was tested to work, so no need to worry).

Replication server:

This is my true favorite. I remember that ASE 15 migration official documents referred to replication server as one of the best methodologies to move from pre-15 to post-15 releases (if I remember right, there was a "Migration Guide" dedicated to moving away from the safety zone of 12.5.x ASEs).  Contrary to that, the installation guides for the ASE 15.7 or ASE 16 do not mentioning replication server alternative for the upgrade.  This is quite unfortunate, since now DBAs who are aware of the possibility to use replication server have to go extra steps to persuade the decision making board as to why this method is the best.  It is not hard to do it (the ASE of origin left intact, downtime is measured in seconds, rollback procedure is clean, rollback is measured in seconds, &c), but had the documentation officially stated this it might have helped DBAs to drive the point.  Actually, since ASE moves to the direction of a full-fledged Replication Server bundled into ASE package (ASE 16 SP02 will have HADR functionality packaged as a part of ASE installation - for those purchasing HADR license) I'd suggest SAPdecision making board to put it back into documentation AND wrap some basic RS functionality into base ASE 16 package restricted to be used for the migration purposes only.  This might have made the migration more "trivial" as well as generated taste for the product for those unfamiliar with it.  Just a thought...

For those employing Rep Server as a part of ASE migration, selecting ASE 15.7 over ASE 16 does not have true advantage points.  The service packs for each are issued more or less at the same rate.  Conservatives might say that ASE 15.7 is more Sybase-flavored while ASE 16 is 100% SAPiant.  But ASE is already 100% SAPiant.  SAP decides where the product will go and it does it not bad at all...

For the conservatives I'd suggest to consider this in their decision making:

AES 16 has over a 1000 bug fixes atop ASE 15.7

Consider the statistics below - based on the release notes for the ASE 16 service patches (Solaris based, Windows have similar number, Linux - may have even more as there are exclusive Linux features in some of future ASE releases).

ASE 16 is packed with more goodies than ASE 15.7

Post SP 100 ASE 15.7 itself is something to strive to (reorg defrag, create index ... online, optimize temp table resolution to avoid SP recompiles, more metrics in monState for those messing with general ASE profiling).  ASE 16 adds even more functionality.  Some of it listed below:

1.  Dynamic Histograms:  useful for tables where out-of-range values are added frequently to the table to avoid costly table scans for new max column values.

2.  Mounting/Unmounting Encrypted Databases:  not possible before ASE 16.

3.  Compressed Indices:  use simple alter database commands to enable/disable index compression (table, index, partition, DOL & APL).

4.  Support for select into #table in multi-statement transaction:  for those who use it - yes, we can.

5.  Partition Locking:  better concurrency mechanisms with partition-level locking.

6.  Wider Query Limit: for subqueries and wild joins, in case it is really necessary.

7.  Create or Replace Semantics:  yes, now in ASE too it is possible to preserve object IDs.

8.  Improved Query Performance:  various improvements in parallelism and access to compressed values.

9.  Full DB Encryption:  a new functionality to complement already available column encryption.

10.Residual Data Removal:  for those sensitive to what may be sniffed out with low level post-DML dbcc commands.

11.Config History:  ability to record config changes in sybsecurity.

12.Wrapping various ELC TFs as config parameter:  Yep, those neat 75x TFs are now configurable.  So is ELC size.

13.ULC Queue:  new way to improve contention on log device - for the nasty "unpin" type contention which was plaguing ULC hit ratio.

14.Select-level with recompile:  ability to recompile only a single SQL statement in SP to assist optimizer.

ASE 16 SP02 will add more features to the list - some pretty heavy weight (like out-of-the box HADR solution, more cache types to keep physical IO as low as possible, dynamic partition management a-la EMC FAST, new ASE management tools - including the precious workload replay capabilities which once DBA might only dream to have - & more).  But that is (close) future.

So which release do you choose?

To put it simple:

If you can go ASE 16, do it.  It has more to offer than ASE 15.7.  If your roll-back procedure is based on Replication Server and thus your migration is instantaneous stopping half-way is not too wise.  If you are note pushed by hard schedules and do not have to commit to the migration project right now you might want to wait for the next ASE 16 release - SP02. It will have an impressive set of functionality to offer.  If the same quarterly release rate continues by SAP,ASE 16 SP02 PL01 will be available in about half a year time.  Not very far ahead. 

If you rely on an ASE binaries based rollback - you don't really have a choice.  ASE 15.7 is your target selection.

Keep in mind, though:  although it may seem that the decision making matrix hinges on the question "do I cross the no-return point?" in fact it does not.  The no-return point will be crossed sooner or later.  Should it be crossed now?  Depends on your readiness and your risk mitigation techniques.

ATM.

I was always wondering what are these new hefty things mean:  ASE Platform edition.... ASE Edge edition.

At last I have found what's in there - and, well, was very pleased.  So much so that I could not resist sharing it at once so it is accessible more easily to others:

I am thrilled that the Developer Edition includes such gems as IMDB, Compression, Semantic Partitioning and Encrypted Column (DB encryption too, I wonder)...  I am thrilled to see the Compression and Semantic Partitioning as a part of both Platform and Edge editions. These are very strong features and since ASE 16 SP02 extends these even further (DSAM functionality) having these options included in base license is fantastic.

I think the Enterprise Edition will at some point become the least attractive ASE Edition.

SAP - you made my day.  I think I'm off for a coffee break ...

ATM

ps.  and by the way, "Runtime Edition" is a nice way to call the all-option-granted ASE on which SAP application run....  Not for sale...

Everyone loves cursors.  Everyone uses cursors.  Not everyone is curious enough to check from time to time what's new in the cursor's world in ASE.  This causes, potentially, funny things to circulate in professional world of ASE developers.

I came across the unpleasant message one day coming from the developers world:

Msg 582, Level 16, State 3:

Server '{SERVER NAME}', Line 1:

Cursor '{CURSOR NAME}' was closed implicitly because the current cursor position was deleted due to an update or a delete.

The cursor scan position could not be recovered. This happens for cursors which reference more than one table.

This is not a pleasant message to receive.  The server has closed your cursor since it has discovered that it's result set has been tampered with. 

Also this message came out in ASE  16 / 15.7, actually this situation is nothing new.  If my memory does not fail me, the earlier releases were more "permissive" to this situation.  They just continued to execute code, skipping rows, doing god knows what.  Only if you was fortunate enough to be able to catch this situation you might have sensed that something is wrong with the flow of your code.  Otherwise you just kept producing bad data unawares.

So, first of all, thank you SAP engineers for closing my cursor.  This is a good message - not a bad message.  This message means that the code I wrote sucks.  A bit.  What is more interesting, though, is that in cases like this developers (or DBAs) will usually go googling or KBAing and if they are lucky, they find some workaround to apply.

As it happened with this message there is a "lucky" number for the curious - SAP Note 2017460 (should I also say "everyone loves SAP notes" to complement my "everyone loves cursors"...  just musing).  KBA goes through a brief reproduction for this situation and comes out with a neat workaround.  I'll skip the reproduction (you may find it here:  https://websmp230.sap-ag.de/sap(bD1lbiZjPTAwMQ==)/bc/bsp/sno/ui_entry/entry.htm?param=69765F6D6F64653D3030312669765F7361…) and jump to the lucky workaround:

Resolution

Use "order by" or "distinct" in the cursor declare.
They makes ASE optimizer use a work table.
That is, cursor position is placed on the work table which avoid error 582.

Wait a minute... Work around?  Order by?  Distinct?  Hmmmm.  That will probably work.  But is it the right way to treat situations like this?  Which release we are talking about?

Yep.  I don't know who is responsible for this KBA but this is a very curios way to engage with cursors.

Since ASE 15.0 new type of cursors were introduced:  scrollable cursors.  Although the scrollable nature of the cursor is not what helps one in this situation (scrollable actually means not that you may walk up and down the cursor but that you may fetch more than one row - including using absolute positioning). Unfortunately, when this new feature got into ASE the other feature that has been added was not given the same amount of focus - it has just been mentioned in passing.  Scrollable cursors may be "insensitive" or "semi-sensitive," or as the NFG stated back then:

"Insensitive" or "semi-sensitive" refers to the extent to which data changes from outside the cursor are visible to the cursor.  A cursor can be semi-sensitive but not scrollable

As to me, the attention must have been reversed.  Few people use scrollable cursor feature.  Some definitely do - otherwise it has not been introduced as a NF.  But all are plagued by the uncanny misbehavior of cursors which sometimes simply do not do what they are supposed to.  At least in the new releases this nasty situation is detected by the server itself and the cursors are closed, issuing error. 

Back to the KBA.  For those using ASE 15.0 and later the way to handle the 582 error is not to tweak around optimizer's access plan and sorting data artificially in order for the server to create a worktable and put the cursor data into it.  The proper way to handle is simply to declare the cursor as "insensitive."  I was very surprised that this has not been put into KBA (should not KBAs be checked by someone before they are released?). 

Now, there might seem to be a caveat to the workaround proposed in the KBA.  Insensitive cursors are.... read-only.  Let's check this:

declare CR_INS insensitive cursor for select name, suid from master..syslogins for update of name

Msg 7301, Level 15, State 1:

FOR UPDATE has been incorrectly specified for a READ ONLY cursor

Hm.  Not nice.  Ok, ok.  I got it.  Silly me.  Let's go back to the KBA workaround:

declare CR_INS cursor for select name, suid from master..syslogins order by suid for update of name

Msg 7301, Level 15, State 1:

FOR UPDATE has been incorrectly specified for a READ ONLY cursor

I rest my point.  Order by does not simply fool the optimizer.  It changes a bunch of other things.

There are two things omitted from the said KBA which I would like to emphasize:

1.  If you face the Msg. 582 situation please re-declare your cursor as insensitive.  I'd say choose this option as a "best practice" - less refactoring in future.  Do not implement the KBA advise.  It is ignorant of the cursor evolution in ASE.  I'd really like to see it removed in future.  Bad coding practice.  Negative performance impact.  Should not be on KBA.

2.  Keep in mind that insensitive cursors are read only (whether they are defined insensitive or turnedinto being insensitive by some tweaking - there is a list in documentation on what turns a cursor into read-only - order by is on that list).

3.  If you have been using the cursor for updating its result set and you hit the Msg. 582... Well... You are in trouble.  You can either cross fingers and let the code run opportunistically (as it all depends on concurrency it will definitely succeed more often than not) or you may put the result set into a #temp table and declare a cursor on it.  Much safer.  Usually works better too. You'll need to check @@rowcount for your updates, though, since those that were hitting 582 may return 0 row affected (not necessarily).

Anyway, the lesson for today:  do not implement KBA blindly. Talk to DBAs first.  Sometimes they have a few things to add.

HTH,

ATM

With increasing speed of business from digitization and other trends, enterprises are faced with greater transaction volume and speed than ever before. ASE 16 SP02 is a timely release that is targeted to address this extreme scale of transaction processing. As part of the ASE 16 release family and following the footsteps of ASE 16.0 that already supports linear scalability of transaction processing workloads on large core-count machines, ASE 16 SP02 takes the extreme online transaction processing (XOLTP) challenge head-on with the following key capabilities, which also address enterprise operational needs for enhanced business continuity and agility:

Business Acceleration

ASE 16 SP02 introduces a new MemScale option that includes new technologies to improve performance with the power of in-memory processing and hardware acceleration. More importantly, no changes are required in existing ASE applications to take advantage of this group of enhancements. The following capabilities are included in the MemScale option:

Simplified Native Access Plan (SNAP) improves the performance of repeatedly run queries, by compiling query plans into native code. The second time an ad-hoc query with a cached plan is executed, ASE generates an intermediate representation that is compiled into native code that provides better performance. Dynamic SQL and stored procedures are also supported, though this capability is currently available on Linux platforms only.

Latch-free B-tree enables users to create indexes that do not require a latch, thereby reducing contention and enabling faster scans of frequently accessed tables. Contention also reduced by use of Transactional Memory on Intel’s latest x86 chipsets (Haswell) in lieu of spinlocks implementing locking and latching mechanisms in ASE. Hardware-enabled transactional memory facility helps to keep track of memory reads/writes more efficiently compared to spinlocks and could provide significant performance gains for workloads that exhibit high lock contention profiles.

Buffer Cache Extension with Flash Memory (to be available in a release after general availability of ASE 16 SP02): To improve I/O performance, ASE makes use of flash storage, which supports higher I/O performance compared to traditional hard drives. ASE’s named caches can be tuned and bound dynamically to flash memory as an intermediate overflow area for buffer cache such that pages can be retrieved from flash memory much faster and dirty buffers can be stored in flash memory and written back to disk in a delayed fashion improving application performance. This is also attractive from cost considerations as Tempdb can be configured to use a combination of smaller size of expensive main memory and a larger, but less expensive, flash memory.

In addition to the new MemScale option, ASE 16 SP02 has enhanced buffer manager technology that helps to reduce cache contention for small tables bound to a named cache. This is enabled by a new setting when creating a cache and it allows multiple threads to access the cache without locking, which improves transaction throughput. This enhancement works on regular tables persisted with on ASE’s page-based store and without changes to the application.

Business Agility

ASE 16 SP02 enables business agility by allowing continuous monitoring, analysis and improvement in handling enterprise transaction workloads. Besides a newly architected ASE Cockpit for monitoring and management, this release introduces a new workload analyzer tool to diagnose and improve ASE performance without materially impacting the production environment.

The new Workload Analyzer option can be used to capture, analyze and replay a workload to ensure that businesses can stay agile and support the needs of changing application requirements. This support includes identifying slow queries and applying configuration changes or other fixes to resolve issues, enable and test new features and compare performance against baseline with same workload, and tune ASE for a given workload without impacting production server, among other benefits.

This release will also help businesses stay agile to the changing data access patterns of the application through support of data tiering using Data Store Access Management, which is part of the base ASE product. This functionality operates on data partitions and provides a facility to define policies and actions to take when policy conditions are met. The policies would typically be associated with an access tier, where the tier is defined by the access pattern of the data partition. For example, a high frequency access pattern on a data partition would characterize it as hot, and by policy the partition could automatically be stored in flash memory devices in uncompressed form for fast access.

This release also introduces a new management and monitoring tool – ASE Cockpit with Database Control Center (DCC)– with simpler alerting mechanism and support for all new ASE features. For details, please see: http://scn.sap.com/community/ase-custom-applications/blog/2015/06/11/whats-new-in-ase-cockpit

Business Continuity

With this release, SAP has integrated its market-leading data replication technology with ASE to provide an innovative, single solution for both High Availability and Disaster Recovery (HADR) that is available as Always-on option in ASE for custom applications. In comparison to storage replication technologies, Always-on provides a low-cost solution that does not require any special clustering software or hardware. The solution is implemented with a pair of database servers, which can serve as HA solution when the primary and companion servers are located within 35Km of each other. For larger distances between the primary and companion servers, configuration for DR is supported. One key difference between HA and DR configurations is that HA supports synchronous replication between primary and companion, thereby ensuring that there is no data loss. For DR configuration, asynchronous replication is supported, which does not guarantee zero data loss. The solution supports automatic fault detection and handling along with transparent client failovers for planned and unplanned failovers.

Some of the key advantages of Always-on that differentiates it from competitive solutions include:

• Zero downtime upgrades – all upgrades and patches can be applied in a rolling fashion without requiring any downtime, unlike storage replication based solutions
• Companion reporting – reporting workloads can be run on the companion node safely with no fear of accidental data modifications. DBAs do not need to take any additional steps to block logins or revoke write privileges from companion node.
• Soft quiesce for planned failovers – avoids termination of in-flight transactions and waits for all active transactions to commit to achieve a soft quiesce state when connections are automatically disconnected with primary and reconnected to companion server. This eliminates the need to patch systems or re-issue transactions that were active when planned failover was initiated.

I am sure this has piqued your interest to learn more about this exciting release of ASE! We are putting together the following series of webinars to provide details of this release. You can register for each session by clicking on the links below, as they become available.

Please watch this posting for additional content pointers in the next couple of months.

For additional information on ASE, see http://www.sap.com/ase