ORA600 blogsLast resort Oracle recoveryhttp://www.ora600.be/blog/atom/feed2008-11-20T10:33:34+01:00Expert Oracle Practiceshttp://www.ora600.be/expert-oracle-practices2010-01-29T11:08:07+01:002010-03-08T00:15:38+01:00kurtvmI was just woken up by the mailman, ringing my doorbell several times.
I jumped out of bed and opened the door half naked and still sleep drunk - I was on call this week and although there weren't many calls I went to bed at 2AM, each day. If there's one thing I hate is going to bed and getting a call an hour later - so I go to bed late at night and hope I don't get a call until 8AM when a colleague takes over.
Anyway, I knew it was the postman delivering the new Oaktable book 'Expert Oracle Practices'.
Because I had bought it at Amazon in the US, I knew I had to pay import taxes (10euro) and thus needed personal delivery. Hey, I wanted to have it asap
This is one of those books you just need to have !
I've already mentioned here that I love going to the UKOUG - and one of the main reasons is because I love to hear how other DBA's/developers do their work. How do they backup their databases, how do they tune their databases, what problems are they having.
The new Oaktable book is very much like that - how do some of the most respected database consultants handle their environments. It's not a low, lower, loooower, Steve Adams low level book. But at first glance it looks more technical than the first Oaktable book (Oracle Insights - Tales of the Oaktable)
And that was also the aim - competing with the other new Oracle 11g books out there.
I'm ashamed to admit that I turned down an offer to write a chapter (even two chapters). I was offered a chapter on backup and recovery and one on NLS issues.
The first one is obvious as I have some experiences with recovery - and I could have written some funny anecdotes and picked out some cases that went bad and why.
But the aim wasn't to write funny stories but technical content.
I have a confession to make - I hate writing technical content/documentation - but I love programming.
If you want to punish me - get me to write technical docs. One of the reasons I quit my previous job was because at one point, they had me writing more technical docs to be ISO9000/1/2 compliant than actually programming stuff. I don't know why I agreed on 2 chapters but with the words of a famous entrepreneur 'screw it, let's do it', I set myself a goal :
try to write 10% of one chapter in one week
Mind you this was the beginning of the summer, the worst possible timing for me, as I had filled up most of the summer weekends with other things to do.
For the backup and recovery chapter, I already had some ideas on how to handle the chapter - I wanted to do a bottom up approach.
Most books do a top down approach - they explain the concept, the wider picture, go to the commands and then some examples, and maybe some internals.
I seem to always learn things bottom up - start with the internals and then work my way up to the commands. It's like a bottom up and top down parser . If you understand the internals, the mechanics, you know what you're doing and thus, don't need to memorize all the commands ... you just need to understand to know what's possible!
Anyway the idea was to first explain block internals, how data is stored and how redo logging works, using a chessboard.
Let's say 2 guys are playing 10 chess games on 10 boards at the same time in the outdoors. They each move pieces, the movement of the pieces get logged by a third person on a piece of paper and once in a while a 4th guy writes down the location of every chess piece on each board to another piece of paper.
The 2 guys playing the chess games are database processes changing rowpieces (chess pieces) in the datablocks (chess board) - the guy writing down the movement of the pieces (redo vectors) is the logwriter process. They guy writing down the complete chessboards to paper is the dbwr process. If it starts to rain, they just scrap the boards (shutdown abort). When they start playing again, they get the paper containing the complete chessboard layouts and the paper with the chess piece movements and redo the movements (instance recovery). It is basic journaling !
With this story in mind I could have explained backup and recovery in a very simple way so everybody understands, and then move my way up.
Anyways - after a week of writing in the evenings I had one page ! One page and I was so bored !!! I'm definitely not a technical writer … a blog now and then…ok. A presentation once a year … fine. But 2 chapters in a book … I don't think so.
So I got back to Jonathan Gennick after a week - I didn't make my goal - I could not see, with the deadlines in place, how I could find time (I have 2 jobs and need to sleep now and then) to write those two chapters and I thought it would be best to let him know before I signed a contract. And you know what, I have not regret it since !
Btw - about the second chapter, NLS issues - I live in Belgium. We have 3 official languages : Dutch, French and German. And we use English to keep everybody happy. So we know allllll about NLS issues
Now - go buy this book - it's awesome !
On a different note …. I do have more time in the winter season and I was able to qualify for the Flemish Poker championship or 'Pokerkampioen Van Vlaanderen'. It's a freeze-out tournament covered by TV. It will be aired on March on 2BE and JimTV. There were 674 entries for the regional finals. I made it through the semi finals and busted out on place 28. I wrote about it here. It's all non-technical content !
]]>I was just woken up by the mailman, ringing my doorbell several times.
I jumped out of bed and opened the door half naked and still sleep drunk - I was on call this week and although there weren't many calls I went to bed at 2AM, each day. If there's one thing I hate is going to bed and getting a call an hour later - so I go to bed late at night and hope I don't get a call until 8AM when a colleague takes over.
Anyway, I knew it was the postman delivering the new Oaktable book 'Expert Oracle Practices'.
Because I had bought it at Amazon in the US, I knew I had to pay import taxes (10euro) and thus needed personal delivery. Hey, I wanted to have it asap
This is one of those books you just need to have !
I've already mentioned here that I love going to the UKOUG - and one of the main reasons is because I love to hear how other DBA's/developers do their work. How do they backup their databases, how do they tune their databases, what problems are they having.
The new Oaktable book is very much like that - how do some of the most respected database consultants handle their environments. It's not a low, lower, loooower, Steve Adams low level book. But at first glance it looks more technical than the first Oaktable book (Oracle Insights - Tales of the Oaktable)
And that was also the aim - competing with the other new Oracle 11g books out there.
I'm ashamed to admit that I turned down an offer to write a chapter (even two chapters). I was offered a chapter on backup and recovery and one on NLS issues.
The first one is obvious as I have some experiences with recovery - and I could have written some funny anecdotes and picked out some cases that went bad and why.
But the aim wasn't to write funny stories but technical content.
I have a confession to make - I hate writing technical content/documentation - but I love programming.
If you want to punish me - get me to write technical docs. One of the reasons I quit my previous job was because at one point, they had me writing more technical docs to be ISO9000/1/2 compliant than actually programming stuff. I don't know why I agreed on 2 chapters but with the words of a famous entrepreneur 'screw it, let's do it', I set myself a goal :
try to write 10% of one chapter in one week
Mind you this was the beginning of the summer, the worst possible timing for me, as I had filled up most of the summer weekends with other things to do.
For the backup and recovery chapter, I already had some ideas on how to handle the chapter - I wanted to do a bottom up approach.
Most books do a top down approach - they explain the concept, the wider picture, go to the commands and then some examples, and maybe some internals.
I seem to always learn things bottom up - start with the internals and then work my way up to the commands. It's like a bottom up and top down parser . If you understand the internals, the mechanics, you know what you're doing and thus, don't need to memorize all the commands ... you just need to understand to know what's possible!
Anyway the idea was to first explain block internals, how data is stored and how redo logging works, using a chessboard.
Let's say 2 guys are playing 10 chess games on 10 boards at the same time in the outdoors. They each move pieces, the movement of the pieces get logged by a third person on a piece of paper and once in a while a 4th guy writes down the location of every chess piece on each board to another piece of paper.
The 2 guys playing the chess games are database processes changing rowpieces (chess pieces) in the datablocks (chess board) - the guy writing down the movement of the pieces (redo vectors) is the logwriter process. They guy writing down the complete chessboards to paper is the dbwr process. If it starts to rain, they just scrap the boards (shutdown abort). When they start playing again, they get the paper containing the complete chessboard layouts and the paper with the chess piece movements and redo the movements (instance recovery). It is basic journaling !
With this story in mind I could have explained backup and recovery in a very simple way so everybody understands, and then move my way up.
Anyways - after a week of writing in the evenings I had one page ! One page and I was so bored !!! I'm definitely not a technical writer … a blog now and then…ok. A presentation once a year … fine. But 2 chapters in a book … I don't think so.
So I got back to Jonathan Gennick after a week - I didn't make my goal - I could not see, with the deadlines in place, how I could find time (I have 2 jobs and need to sleep now and then) to write those two chapters and I thought it would be best to let him know before I signed a contract. And you know what, I have not regret it since !
Btw - about the second chapter, NLS issues - I live in Belgium. We have 3 official languages : Dutch, French and German. And we use English to keep everybody happy. So we know allllll about NLS issues
Now - go buy this book - it's awesome !
On a different note …. I do have more time in the winter season and I was able to qualify for the Flemish Poker championship or 'Pokerkampioen Van Vlaanderen'. It's a freeze-out tournament covered by TV. It will be aired on March on 2BE and JimTV. There were 674 entries for the regional finals. I made it through the semi finals and busted out on place 28. I wrote about it here. It's all non-technical content !
]]>UKOUG2009 - I too can blog about it ...http://www.ora600.be/ukoug-20092009-12-03T15:24:27+01:002010-03-08T00:16:15+01:00kurtvmI don't know how they do it !
When I go to a conference, I make an agenda and mostly follow it all the way through - I might make a few changes here or there, but I won’t easily skip a slot… unless an emergency comes up … which happened only twice in these 3 days
That brings me to my first observation of UKOUG2009 … wifi sucked !!!
When I go to a conference I still need to be available – I *need*wifi access to survive. I only was able to get the ICC wifi to work during the parties!
The mickey mouse portables available at the exibition hall and registration area didn’t do me any good either – I couldn’t ssh out of them !
Luckily I found some unprotected wifi signals in the exhibition area – my gratitude goes out to the people who had setup the NETGEAR wifi appliance – you delivered excellent internet services although I reckon this was unintentionally as it was protected the last day of the conference
Here ‘s a way to get people to come to your booth – if you have wifi – name the network to your company, and give people the password if they visit you – success guaranteed !
Anyway – I got deferred – I was wondering how people do it –blogging and twittering during a conference while there is so much to do, so much to see, so much to process. And I reckon, a lot of people were twitteringand blogging … that probably was the cause for the bad wifi !!!
So I decided to blog a bit about UKOUG2009 when I got home.
We arrived on Sunday at Jurys Inn – which is about the best hotel you can get without going to the Hyatt. Every year I seem to bring mor eand more people of the company with me – this year our company had sent 9 people ! The hotel looked a bit quiet so we decided to head to All-bar-one.Which again looked a bit deserted … it seems the credit crunch had taken it’s toll here !
So what about the sessions – I have a broad interest – the last couple of years I do more and more application server stuff, networking, identity management and programming than hardcore DBA work.
I had sessions on my agenda ranging from ‘Getting the best out of hardware loadbalancers’ to ‘Integrating forms with apex’ over ‘Authentication,SSO and authorization for WLS’ and Tanel’s ‘Latches and mutexes’ talk. And they say I am a database geek – I reckon I attend more talks on middleware thandatabases.
Now for those of you moaning about how bad some presentations were and how you could do so much better – go out there and do it. Write a apresentation and/or whitepaper and get your butt out there.
One of my colleagues attended a talk on ADF 11g by someonefrom Oracle – afterwards they had a chat and in between my colleague mentions how his team had created a kick ass app in ADF 11g. In which he got the reply ‘well– show it to me’ ! End result – if he writes a paper about it, he may present it at Oracle Openworld 2010!
So what about the sessions I attended – here are some highlights :
Alex Keh’s talk on Active Directory and Windows Security integration with Oracle databases – Alex presentation was great – excellent topic, good presentation style, working demos and I learned a couple of new things.
Joel Goodman from Oracle education had a strange presentation title – something about DB Links PART 2. I couldn’t find part 1 in the agenda. Apparently he had submitted 2 presentations but part 1 didn’t get accepted. Nonetheless – Joel Goodman knows his stuff – he also reminds me of Wolverine – deep voice, American accent … it’s almost as if he tries to hypnotize you while pumping valuable knowledge into your brain. (I wouldn’t mind having him around in a street fight ;-) ) This presentation was all about distributed transactions and how crashing databases, involved in distributed transaction,can lock complete tables until you force commit or force rollback the transactions. I had completely forgotten about that. The last time I had to do something like that was in the Oracle7 era. So 2 thumbs up ! Joel puts hispresentations free for downloading here: http://dbatrain.wordpress.com/articles-papers-and-presentations
Joel’s partner in crime Harald Van Breederode, and equally excellent presenter talked about 11g SQL Plan Management. This is something I get confronted with quite a lot – plan stability. Don’t you hate it when a new execution plan just hogs down a complete database. In my opinion, Oracle should have introduced SQL Plan Management when they introduced the CBO ! The one can’t live without the other and yet it has taken over a decade to get this feature –my guess is it was just too resource hungry for the hardware available back in the days. I love to quote Hannibal Smith on this : ‘I love it when a plan comes together!’ Harald hasn’t posted his presentation on his blog – but who knows, one day he might!
I’m a java guy but that doesn’t mean I can’t appreciate plsql – I like it better then say … perl – so I went to see Steven Feuerstein’s‘High Performance PL/SQL’ – very interesting stuff. He spent quite some time on ways to cache data in the pga/uga using all sorts of tricks, ending up with 11g’s result cache. Not too bad I thought – but Connor McDonald showed a little bit more skepticisms on the result cache. He proofed that if the result cache isbeing populated and concurrent sessions are running the same query which ispopulating the cache, extra locking occurs. As usual, Connor delivered one of the best, well probably most definitely *the* best presentation of UKOUG2009 inhis usual style, firing away *485* slides in 45 minutes !
I went to quite a lot of weblogic/ias presentations. It looks like people that are Oracle acquired from the acquisitions haven’t the slightest idea of what Oracle Portal/Forms/Reports are, and *how many* people are still using it. Every presentation was heavily (not heavenly) focused on java functionality, with Forms/reports dangling, almost falling of the slides ! One guy was quite hilarious – it was a talk about SSO and WLS … to be honest I was expecting something about how ‘legacy’ apps like Oracle Forms/reports/portal and Oracle SSO (OC4J_Security) would integrate with the new Weblogic App server. Instead it was a talk about Oracle’s Identity & Access Management suite build on top of WLS. No biggy – quite interesting especially from someone who co-wrote the SAML standards.
But what I suspected would happen, did happen … someone asked a question about how all this would integrate with Oracle Portal if they migrate from IAS 10.1.2.3 to Weblogic ……………….
This was going to be good as I was also interested in that. The answer was way off, it was quite clear he had no idea what Oracle Portal was or how it is deeply integrate with PLSQL (and the PLSQL SSO api) and OID. The person who asked the question tried to reframe the question but again … no serious answer –‘no, but in java … blah blah blah’ … you could see the disappointment as the person who asked the question politely nodded along and thanked the presenter for this very straightforward answer.
This confirms to me that Portal is *dead*. If you run a public website on Portal – get drupal. It’s cheap, it has way better SEO and thrives on a very large community. Oh – and I have created a drupal module that integrates drupal user management with Oracle SSO. Throw away Portal – it’s a real pain to migrate upwards (I’m stuck at Portal 10.1.2.3 as I’m unable to migrate to 10.1.4.2 even after a 2 week SR with Oracle). Throw it out… you’ll feel much better and people will like you for it
One other hilarious moment was Julian Dyke’s presentation on 11gR2 new features. Julian makes excellent presentation/slides/animations. So for that reason alone, if I can, I tend to go to any of his talks. However, at the beginning of his presentation, he mentioned he had forgotten a word in his title : ‘RAC’ – ‘11gR2 *RAC* new features’. This resulted in quite a lot of people leaving the room. I decided to stay purely based on Julian’s reputation.However, I had to leave afterwards when I got an important phone call …(damn wifi) and I was glad I had an excuse to leave the room. I took the 11g RAC course in February of this year. When Julian started to talk about all the new RAC stuff like SCAN, I felt depressed. It was like I could throw away half thet hings I had learned in the course – ‘this is new, that’s new, that’s changed,this is obsolete, this works like that now …’ … too depressing …
To end this post – here’s a picture of a collegue of mine at the Fire & Ice Party (disclaimer – the girl had already left – she’s not under the table)
I need to lay down now after another great UKOUG conference .... till next year !!!!
]]>I don't know how they do it !
When I go to a conference, I make an agenda and mostly follow it all the way through - I might make a few changes here or there, but I won’t easily skip a slot… unless an emergency comes up … which happened only twice in these 3 days
That brings me to my first observation of UKOUG2009 … wifi sucked !!!
When I go to a conference I still need to be available – I *need*wifi access to survive. I only was able to get the ICC wifi to work during the parties!
The mickey mouse portables available at the exibition hall and registration area didn’t do me any good either – I couldn’t ssh out of them !
Luckily I found some unprotected wifi signals in the exhibition area – my gratitude goes out to the people who had setup the NETGEAR wifi appliance – you delivered excellent internet services although I reckon this was unintentionally as it was protected the last day of the conference
Here ‘s a way to get people to come to your booth – if you have wifi – name the network to your company, and give people the password if they visit you – success guaranteed !
Anyway – I got deferred – I was wondering how people do it –blogging and twittering during a conference while there is so much to do, so much to see, so much to process. And I reckon, a lot of people were twitteringand blogging … that probably was the cause for the bad wifi !!!
So I decided to blog a bit about UKOUG2009 when I got home.
We arrived on Sunday at Jurys Inn – which is about the best hotel you can get without going to the Hyatt. Every year I seem to bring mor eand more people of the company with me – this year our company had sent 9 people ! The hotel looked a bit quiet so we decided to head to All-bar-one.Which again looked a bit deserted … it seems the credit crunch had taken it’s toll here !
So what about the sessions – I have a broad interest – the last couple of years I do more and more application server stuff, networking, identity management and programming than hardcore DBA work.
I had sessions on my agenda ranging from ‘Getting the best out of hardware loadbalancers’ to ‘Integrating forms with apex’ over ‘Authentication,SSO and authorization for WLS’ and Tanel’s ‘Latches and mutexes’ talk. And they say I am a database geek – I reckon I attend more talks on middleware thandatabases.
Now for those of you moaning about how bad some presentations were and how you could do so much better – go out there and do it. Write a apresentation and/or whitepaper and get your butt out there.
One of my colleagues attended a talk on ADF 11g by someonefrom Oracle – afterwards they had a chat and in between my colleague mentions how his team had created a kick ass app in ADF 11g. In which he got the reply ‘well– show it to me’ ! End result – if he writes a paper about it, he may present it at Oracle Openworld 2010!
So what about the sessions I attended – here are some highlights :
Alex Keh’s talk on Active Directory and Windows Security integration with Oracle databases – Alex presentation was great – excellent topic, good presentation style, working demos and I learned a couple of new things.
Joel Goodman from Oracle education had a strange presentation title – something about DB Links PART 2. I couldn’t find part 1 in the agenda. Apparently he had submitted 2 presentations but part 1 didn’t get accepted. Nonetheless – Joel Goodman knows his stuff – he also reminds me of Wolverine – deep voice, American accent … it’s almost as if he tries to hypnotize you while pumping valuable knowledge into your brain. (I wouldn’t mind having him around in a street fight ;-) ) This presentation was all about distributed transactions and how crashing databases, involved in distributed transaction,can lock complete tables until you force commit or force rollback the transactions. I had completely forgotten about that. The last time I had to do something like that was in the Oracle7 era. So 2 thumbs up ! Joel puts hispresentations free for downloading here: http://dbatrain.wordpress.com/articles-papers-and-presentations
Joel’s partner in crime Harald Van Breederode, and equally excellent presenter talked about 11g SQL Plan Management. This is something I get confronted with quite a lot – plan stability. Don’t you hate it when a new execution plan just hogs down a complete database. In my opinion, Oracle should have introduced SQL Plan Management when they introduced the CBO ! The one can’t live without the other and yet it has taken over a decade to get this feature –my guess is it was just too resource hungry for the hardware available back in the days. I love to quote Hannibal Smith on this : ‘I love it when a plan comes together!’ Harald hasn’t posted his presentation on his blog – but who knows, one day he might!
I’m a java guy but that doesn’t mean I can’t appreciate plsql – I like it better then say … perl – so I went to see Steven Feuerstein’s‘High Performance PL/SQL’ – very interesting stuff. He spent quite some time on ways to cache data in the pga/uga using all sorts of tricks, ending up with 11g’s result cache. Not too bad I thought – but Connor McDonald showed a little bit more skepticisms on the result cache. He proofed that if the result cache isbeing populated and concurrent sessions are running the same query which ispopulating the cache, extra locking occurs. As usual, Connor delivered one of the best, well probably most definitely *the* best presentation of UKOUG2009 inhis usual style, firing away *485* slides in 45 minutes !
I went to quite a lot of weblogic/ias presentations. It looks like people that are Oracle acquired from the acquisitions haven’t the slightest idea of what Oracle Portal/Forms/Reports are, and *how many* people are still using it. Every presentation was heavily (not heavenly) focused on java functionality, with Forms/reports dangling, almost falling of the slides ! One guy was quite hilarious – it was a talk about SSO and WLS … to be honest I was expecting something about how ‘legacy’ apps like Oracle Forms/reports/portal and Oracle SSO (OC4J_Security) would integrate with the new Weblogic App server. Instead it was a talk about Oracle’s Identity & Access Management suite build on top of WLS. No biggy – quite interesting especially from someone who co-wrote the SAML standards.
But what I suspected would happen, did happen … someone asked a question about how all this would integrate with Oracle Portal if they migrate from IAS 10.1.2.3 to Weblogic ……………….
This was going to be good as I was also interested in that. The answer was way off, it was quite clear he had no idea what Oracle Portal was or how it is deeply integrate with PLSQL (and the PLSQL SSO api) and OID. The person who asked the question tried to reframe the question but again … no serious answer –‘no, but in java … blah blah blah’ … you could see the disappointment as the person who asked the question politely nodded along and thanked the presenter for this very straightforward answer.
This confirms to me that Portal is *dead*. If you run a public website on Portal – get drupal. It’s cheap, it has way better SEO and thrives on a very large community. Oh – and I have created a drupal module that integrates drupal user management with Oracle SSO. Throw away Portal – it’s a real pain to migrate upwards (I’m stuck at Portal 10.1.2.3 as I’m unable to migrate to 10.1.4.2 even after a 2 week SR with Oracle). Throw it out… you’ll feel much better and people will like you for it
One other hilarious moment was Julian Dyke’s presentation on 11gR2 new features. Julian makes excellent presentation/slides/animations. So for that reason alone, if I can, I tend to go to any of his talks. However, at the beginning of his presentation, he mentioned he had forgotten a word in his title : ‘RAC’ – ‘11gR2 *RAC* new features’. This resulted in quite a lot of people leaving the room. I decided to stay purely based on Julian’s reputation.However, I had to leave afterwards when I got an important phone call …(damn wifi) and I was glad I had an excuse to leave the room. I took the 11g RAC course in February of this year. When Julian started to talk about all the new RAC stuff like SCAN, I felt depressed. It was like I could throw away half thet hings I had learned in the course – ‘this is new, that’s new, that’s changed,this is obsolete, this works like that now …’ … too depressing …
To end this post – here’s a picture of a collegue of mine at the Fire & Ice Party (disclaimer – the girl had already left – she’s not under the table)
I need to lay down now after another great UKOUG conference .... till next year !!!!
]]>Cloud computing - my server is out there man !http://www.ora600.be/my-server-was-out-there-in-the-cloud2009-11-16T10:16:27+01:002009-11-16T10:22:44+01:00kurtvmPrice is non-negotiable
This website is hosted on one of my servers co-located at a data centre near Brussels. It's a DELL Poweredge 850E with 340W powersupply.
I use my own dedicated server because I *sometimes* need large amount of storage. For example, if a customer has a crashed database and wants to upload his datafiles. We're speaking 100Gb or more.
So I need large amounts of storage - not all the time - and bandwidth. The server is hooked up to a 100Mbit internet backbone so I got that one covered too !
A couple of years ago, my only option was to buy a physical server, find a hosting company that offers co-location and hire 1U rackspace.
Cloud computing has changed this game - that and the price of electricity at datacenters.
Two months ago I was thinking of buying a server to replace the old DELL server - it was 3y old and I wanted bigger disks
Coincidentally, when I had configured a new 1U box on DELL's website, I got a call from my hosting company.
After hearing a whole story about how they got new racks (more current, better cooling, bla bla bla) and that I had to migrate my server, they dropped the bombshell on pricing.
Electricity prices had gone up and they had to re-negotiate prices with the datacenter.
Much to my surprise, the new price was three times the current price. Now I don't know about you - but if you have to pay 3x the price for getting the same service, something is wrong (did I mention that the year before they also upped their price with 20%)
Suddenly a power supply of 340W became extremely expensive. Did you ever had a look at those 200W servers … I'd rather run a business on a laptop than on one of those mickey-mouse servers… at least my old DELL has hardware raid and a decent cpu...
So - either they were lousy negotiators - or they wanted their customers to pay for their new - probably golden - racks.
It was a lousy sales-pitch too - the sales guy offered me to think about it and check out the competition … which were all cheaper than them ;-)
Anyway - prices were non-negotiable - so I first checked out several competitors in the same and other datacenters.
Now with co-location - all you get is rackspace, current and network. That's it.
One hosting provider's current is not better than another's - so the only criteria are price and maybe the datacenter's reputation. (well ok - there are minor differences regarding network limits, but co-location is basically easy money for providers).
So there I was - comparing prices of different providers at different datacenters. And as price and network was my only criteria I chose to move to a small provider at the Scarlet datacenter in Vilvoorde. It's a really small and young provider/company as they only have 2 racks there (and 2 racks at another datacenter).
Now, I didn't want any downtime - that meant renting computing power, temporarely migrating the website and all programs that allow for the generating DUDE licenses (which is a bit more complicated that running some 'make' or 'ant' scripts), switching DNS, while doing the physical transfer of the server.
The Anjo connection
I remembered one time I was at Anjo Kolk's place. I had bought 2 second hand Sun servers from him. He had bought 40 of them in the US and had shipped them to The Netherlands, so he could spare a server or two
Anyway - I told him I was going to use the servers for a new project of mine, where I needed a linux cluster for running my own DNS servers. At which point he started demonstrating Amazon EC2 and how I could use cloud computing to do what I wanted to do, without the hassles of hiring extra rackspace or getting a NAS for storage.
I must say I was intrigued … but it all looked a bit fragile and over-complicated.
I mean, I just wanted some cpu, some storage and network.
With Amazon I had to choose from an aws image in the EC2 cloud - then, if I wanted persistent storage, I had to rent Amazon S3 storage. Huh persistent storage ?
It looks like Amazon's VM's are snapshots from common images and if you shutdown your VM or your VM crashes, the VM reverts to the original image, losing all your changes… unless you backed it up to your persistent S3 storage … which couldn't be mounted as a file systems but had to accessed through HTTP protocols … huh???
I just want disk space … how hard is that ?
Anyway - because of migration and uptime issues I was facing, I took a new look at Amazon EC2 and it seems they finally have mountable persistent storage, which they call Amazon EBS (or elastic block device) or what I like to call - a disk.
Still, if your VM crashes, you lose your configuration unless you back it up to your EBS or somewhere else and reconfigure after the crash (someone correct me if I'm wrong).
That's OK as I only needed a VM for a day or 2, while my own server was physically being moved to the new datacenter. So I signed up for the Amazon EC2 service. And then I waited … and waited…. It took them 2 days to activate my account… too slow.
Amazon and goliath
While I was waiting for my account activation at Amazon, I decided to look at other cloud providers … there is only a handful and most of them are based on virtualization software like VMware, Xen and KVM. The downside is that they all offer fixed configurations like :
Small config : 1 cpu core, 1Gb mem, 20Gb disk
Medium config : 2 cpu cure, 2Gb mem, 40Gb disk
Large …. You get the idea
In my opinion - cloud computing should provide flexible configurations at flexible prices.
This was only cloud computing in the sense that you pay by the hour but it's not as flexible as Amazon that you can say - I want 1 cpu core, 1Gb of memory and 200Gb of disk.
Nooo, if you wanted 200Gb of disk space, you needed to take the *extra large config* with 16 cpu cores, 8Gb memory and that happened to come with 200Gb of disk space and will cost you as much as a half a rack and 4 DELLs if you keep running it for a whole year.
But wait - I've found the one exception, that absolutely lets you choose all 3 (cpu/mem/storage) independently - and they are called Elastichosts.com, a UK based cloud provider.
And what's more - they let you try it out for free for 5 days !
And … they give you a fixed IP address unlike Amazon's elastic IP address which will make you jump to 20 burning loops before you can receive/send mail to/from your server that is not marked as spam.
Elastichosts runs on Linux KVM and they have made sure that all interfaces are very simple - a disk is a disk - an static ip really is a static ip.
No fuzzy acronyms.
In fact, I was reminded of Larry Ellison's opinion of cloud computing … it's just a simple server that is 'out there' - it needs cpu, memory, network and electricity - and for me affordable flexible pricing - that's it !!!
Maybe I'm an idiot but Amazon just took it a step too far and made it too complex !
In case you've misted Larry's interview on cloud computing :
Epilogue
Anjo has been running an Amazon AWS for over a year without crashing.
I migrated the ORA600 server without downtime to the Scarlet datacenter which is located in the same building as the Belgian Oracle office (you've got to love the irony )
A friend of mine who was also co-locating a server at my original provider was offered negotiable prices a couple of weeks later (maybe tripling their prices wasn't the best of ideas). He too migrated to another datacenter.
If a hosting company ever pulls a trick on me again, I'm throwing away my hardware and move to elastichosts.com for good.
This website is hosted on one of my servers co-located at a data centre near Brussels. It's a DELL Poweredge 850E with 340W powersupply.
I use my own dedicated server because I *sometimes* need large amount of storage. For example, if a customer has a crashed database and wants to upload his datafiles. We're speaking 100Gb or more.
So I need large amounts of storage - not all the time - and bandwidth. The server is hooked up to a 100Mbit internet backbone so I got that one covered too !
A couple of years ago, my only option was to buy a physical server, find a hosting company that offers co-location and hire 1U rackspace.
Cloud computing has changed this game - that and the price of electricity at datacenters.
Two months ago I was thinking of buying a server to replace the old DELL server - it was 3y old and I wanted bigger disks
Coincidentally, when I had configured a new 1U box on DELL's website, I got a call from my hosting company.
After hearing a whole story about how they got new racks (more current, better cooling, bla bla bla) and that I had to migrate my server, they dropped the bombshell on pricing.
Electricity prices had gone up and they had to re-negotiate prices with the datacenter.
Much to my surprise, the new price was three times the current price. Now I don't know about you - but if you have to pay 3x the price for getting the same service, something is wrong (did I mention that the year before they also upped their price with 20%)
Suddenly a power supply of 340W became extremely expensive. Did you ever had a look at those 200W servers … I'd rather run a business on a laptop than on one of those mickey-mouse servers… at least my old DELL has hardware raid and a decent cpu...
So - either they were lousy negotiators - or they wanted their customers to pay for their new - probably golden - racks.
It was a lousy sales-pitch too - the sales guy offered me to think about it and check out the competition … which were all cheaper than them ;-)
Anyway - prices were non-negotiable - so I first checked out several competitors in the same and other datacenters.
Now with co-location - all you get is rackspace, current and network. That's it.
One hosting provider's current is not better than another's - so the only criteria are price and maybe the datacenter's reputation. (well ok - there are minor differences regarding network limits, but co-location is basically easy money for providers).
So there I was - comparing prices of different providers at different datacenters. And as price and network was my only criteria I chose to move to a small provider at the Scarlet datacenter in Vilvoorde. It's a really small and young provider/company as they only have 2 racks there (and 2 racks at another datacenter).
Now, I didn't want any downtime - that meant renting computing power, temporarely migrating the website and all programs that allow for the generating DUDE licenses (which is a bit more complicated that running some 'make' or 'ant' scripts), switching DNS, while doing the physical transfer of the server.
The Anjo connection
I remembered one time I was at Anjo Kolk's place. I had bought 2 second hand Sun servers from him. He had bought 40 of them in the US and had shipped them to The Netherlands, so he could spare a server or two
Anyway - I told him I was going to use the servers for a new project of mine, where I needed a linux cluster for running my own DNS servers. At which point he started demonstrating Amazon EC2 and how I could use cloud computing to do what I wanted to do, without the hassles of hiring extra rackspace or getting a NAS for storage.
I must say I was intrigued … but it all looked a bit fragile and over-complicated.
I mean, I just wanted some cpu, some storage and network.
With Amazon I had to choose from an aws image in the EC2 cloud - then, if I wanted persistent storage, I had to rent Amazon S3 storage. Huh persistent storage ?
It looks like Amazon's VM's are snapshots from common images and if you shutdown your VM or your VM crashes, the VM reverts to the original image, losing all your changes… unless you backed it up to your persistent S3 storage … which couldn't be mounted as a file systems but had to accessed through HTTP protocols … huh???
I just want disk space … how hard is that ?
Anyway - because of migration and uptime issues I was facing, I took a new look at Amazon EC2 and it seems they finally have mountable persistent storage, which they call Amazon EBS (or elastic block device) or what I like to call - a disk.
Still, if your VM crashes, you lose your configuration unless you back it up to your EBS or somewhere else and reconfigure after the crash (someone correct me if I'm wrong).
That's OK as I only needed a VM for a day or 2, while my own server was physically being moved to the new datacenter. So I signed up for the Amazon EC2 service. And then I waited … and waited…. It took them 2 days to activate my account… too slow.
Amazon and goliath
While I was waiting for my account activation at Amazon, I decided to look at other cloud providers … there is only a handful and most of them are based on virtualization software like VMware, Xen and KVM. The downside is that they all offer fixed configurations like :
Small config : 1 cpu core, 1Gb mem, 20Gb disk
Medium config : 2 cpu cure, 2Gb mem, 40Gb disk
Large …. You get the idea
In my opinion - cloud computing should provide flexible configurations at flexible prices.
This was only cloud computing in the sense that you pay by the hour but it's not as flexible as Amazon that you can say - I want 1 cpu core, 1Gb of memory and 200Gb of disk.
Nooo, if you wanted 200Gb of disk space, you needed to take the *extra large config* with 16 cpu cores, 8Gb memory and that happened to come with 200Gb of disk space and will cost you as much as a half a rack and 4 DELLs if you keep running it for a whole year.
But wait - I've found the one exception, that absolutely lets you choose all 3 (cpu/mem/storage) independently - and they are called Elastichosts.com, a UK based cloud provider.
And what's more - they let you try it out for free for 5 days !
And … they give you a fixed IP address unlike Amazon's elastic IP address which will make you jump to 20 burning loops before you can receive/send mail to/from your server that is not marked as spam.
Elastichosts runs on Linux KVM and they have made sure that all interfaces are very simple - a disk is a disk - an static ip really is a static ip.
No fuzzy acronyms.
In fact, I was reminded of Larry Ellison's opinion of cloud computing … it's just a simple server that is 'out there' - it needs cpu, memory, network and electricity - and for me affordable flexible pricing - that's it !!!
Maybe I'm an idiot but Amazon just took it a step too far and made it too complex !
In case you've misted Larry's interview on cloud computing :
Epilogue
Anjo has been running an Amazon AWS for over a year without crashing.
I migrated the ORA600 server without downtime to the Scarlet datacenter which is located in the same building as the Belgian Oracle office (you've got to love the irony )
A friend of mine who was also co-locating a server at my original provider was offered negotiable prices a couple of weeks later (maybe tripling their prices wasn't the best of ideas). He too migrated to another datacenter.
If a hosting company ever pulls a trick on me again, I'm throwing away my hardware and move to elastichosts.com for good.
]]>ORA600 Aggregator mobile themehttp://www.ora600.be/node/53482009-10-09T09:47:20+02:002009-10-09T09:48:57+02:00kurtvm
I caved in ... I bought an iPhone.
I still think the price is a total rip-off here in Belgium - and there's a 6 week waiting list.
Anyway - when I looked at the news aggregator, I thought I could do better and so the aggregator now detects mobile browsers and switches to a mobile theme. The implementation uses an mobile theme, a browser detection (browscap), a themeswitcher and the iWebKit. The implementation was done in 15min all because of the drupal framework . I wonder how long this would take in Apex - or god forbid Portal !
When I started this site, and needed to pick a framework, I was favoring Apex because I knew it already. And I have some experience with Oracle Portal. However, the problem we were having at a customer site with Portal (or apex) and search engine optimisation (SEO), drove me into drupal which does a great job in SEO. It did involve learning php (but I already knew perl really well) and ofcourse the drupal hook api (which is really another way of thinking and programming). But the payoff comes with all free modules that you can find on the internet and build new functionality in a matter of minutes !!!
]]>
I caved in ... I bought an iPhone.
I still think the price is a total rip-off here in Belgium - and there's a 6 week waiting list.
Anyway - when I looked at the news aggregator, I thought I could do better and so the aggregator now detects mobile browsers and switches to a mobile theme. The implementation uses an mobile theme, a browser detection (browscap), a themeswitcher and the iWebKit. The implementation was done in 15min all because of the drupal framework . I wonder how long this would take in Apex - or god forbid Portal !
When I started this site, and needed to pick a framework, I was favoring Apex because I knew it already. And I have some experience with Oracle Portal. However, the problem we were having at a customer site with Portal (or apex) and search engine optimisation (SEO), drove me into drupal which does a great job in SEO. It did involve learning php (but I already knew perl really well) and ofcourse the drupal hook api (which is really another way of thinking and programming). But the payoff comes with all free modules that you can find on the internet and build new functionality in a matter of minutes !!!
]]>Oracle Performance Firefightinghttp://www.ora600.be/node/48822009-09-10T21:23:58+02:002009-09-10T21:55:37+02:00kurtvmI've been - and still am - very busy lately.
I'm a bottom-up learner - like bottom-up parsers (as opposed to top-down parsers) I like to get my facts as detailed as possible and then work my way up to more general topics. For example, when I learned my backup & recovery skills, I learned about block internals first as opposed to diving in all sorts of high level commands.
I found out about the book a couple of months ago as Daniel Fink told me he was reviewing this book - I immediately ordered it on Craig's site before it was available on Amazon.
I must admit, I've only read chapters 6, 7 and 8 - covering buffer cache internals, redo internals and shared pool internals. These are 3 chapters every dba should read !!! Those 3 chapters are worth the price on their own !
So if you're buying only one Oracle book this year - buy this one !!!
I'm a bottom-up learner - like bottom-up parsers (as opposed to top-down parsers) I like to get my facts as detailed as possible and then work my way up to more general topics. For example, when I learned my backup & recovery skills, I learned about block internals first as opposed to diving in all sorts of high level commands.
I found out about the book a couple of months ago as Daniel Fink told me he was reviewing this book - I immediately ordered it on Craig's site before it was available on Amazon.
I must admit, I've only read chapters 6, 7 and 8 - covering buffer cache internals, redo internals and shared pool internals. These are 3 chapters every dba should read !!! Those 3 chapters are worth the price on their own !
So if you're buying only one Oracle book this year - buy this one !!!
]]>Oracle buys sun ... time to buy some Oracle shareshttp://www.ora600.be/oracle-buys-sun2009-04-20T14:03:35+02:002009-08-23T16:22:58+02:00kurtvm
It seems Oracle succeeded where IBM failed :
http://www.oracle.com/sun/index.html
"On April 20, 2009, Oracle announced it has entered into an agreement to acquire Sun Microsystems (Sun). The proposed transaction is subject to Sun stockholder approval, certain regulatory approvals and customary closing conditions. Until the deal closes, each company will continue to operate independently, and it is business as usual.
The acquisition combines best-in-class enterprise software and mission-critical computing systems. Oracle plans to engineer and deliver an integrated system—applications to disk—where all the pieces fit and work together so customers do not have to do it themselves. Customers benefit as their system integration costs go down while system performance, reliability and security go up.
Investor Conference Call
There will be a conference call to discuss the transaction at 5:30 a.m. Pacific time.
Investors can listen to the conference call by dialing +1.719.234.7870, passcode: 923645
A replay will be available for 24 hours after the call ends at +1.719.884.8882, passcode: 923645"
]]>
It seems Oracle succeeded where IBM failed :
http://www.oracle.com/sun/index.html
"On April 20, 2009, Oracle announced it has entered into an agreement to acquire Sun Microsystems (Sun). The proposed transaction is subject to Sun stockholder approval, certain regulatory approvals and customary closing conditions. Until the deal closes, each company will continue to operate independently, and it is business as usual.
The acquisition combines best-in-class enterprise software and mission-critical computing systems. Oracle plans to engineer and deliver an integrated system—applications to disk—where all the pieces fit and work together so customers do not have to do it themselves. Customers benefit as their system integration costs go down while system performance, reliability and security go up.
Investor Conference Call
There will be a conference call to discuss the transaction at 5:30 a.m. Pacific time.
Investors can listen to the conference call by dialing +1.719.234.7870, passcode: 923645
A replay will be available for 24 hours after the call ends at +1.719.884.8882, passcode: 923645"
]]>FBI internal investigations...hidden virtual columns in 8i.http://www.ora600.be/function-based-index-virtual-column2009-02-23T08:51:35+01:002009-02-23T09:06:59+01:00kurtvm
I was called in last week for a performance issue at a customer site.
Let's set scene a bit …
They had moved their main production OLTP database from AIX to Windows and had been struggling with performance for a couple of weeks. Hardware-wise, they went from expensive but old Bull hardware to a Dell box (with 8 cores). And they went from 4Gb to 16Gb RAM. So you would expect the database would fly on this new hardware.
Now - this database was/is pulling around a 1000 concurrent users…. The first thing they ran into was the process' addressable memory space on 32bit windows, being 2Gb. This of course led to a lot of ORA-04030 and angry users. Actually, Anjo recently made a blogentry about it.
So actions were made to optimize Oracle for windows, which basically consists of putting the buffercache above the 2Gb limit - anyway metalink note 231159.1 explains all of this, but still, as this is an Oracle 8.1.7.4 database handling a thousand users, all other memory (remember sort_area_size, hash_area_size etc) related parameters were set to a minimum to avoid ORA-4030's.
The result of this memory balancing operation was, that performance was below average. And I was send in to have a look at it. I must admit - I hadn't tuned a whole environment for a while, but I had done a lot of 8.0 and 7.3 migrations to 8i back in the day, and knew pretty much what the problems of the 8i CBO were. Oh - why are the still running 8i … well the database application is to be decommissioned this year and so they didn't bother to move up to the latest Oracle version. The fact that they wanted to invest in getting performance up, proves power users were about to lynch the IT manager
Anyway - 5minutes into the meeting with IT, a second consultant from a rivaling company walks in. That's how bad performance was. I don't mind competition - in fact I didn't see it as competition. The guy was more than reasonable and during the day we acted as each-others soundboard.
The first thing I noticed were the bad response times on the disks and the large amount of physical IO the database was doing. The database running on a Dell SAN…. I didn't even know Dell produced SANs !!! :-p But I knew one thing - this SAN was sh*te. It used 8 disks in RAID5 (coming from 4 disks) and I reckon the write cache must have been quite small. In fact at first I thought the SAN cache must have been disabled - something I had encountered multiple times when the battery of the cache dies, the cache on the SAN is disabled to avoid corruption in case of power-failure.
Clearly, this DB was doing too much IO for the SAN… way past it's critical point in the hockey stick chart see also Anjo's paper on this - oh and we couldn't increase any memory settings within the DB of course.
So the only option left, was hunting down individual sql statements and tune them, so the overall IO would decrease. The impact of sql tuning can be *huge* - especially with a 1000 oltp users. Just imaging a sql that does 5000 physical IO's that is run by these 1000 users every minute (5x10^6 IO/min) - if we can tune it to use only 20 physical IO's (20x10^3 IO/min) than that would have a dramatic effect on overall performance.
I noticed that there were a lot of full tablescans … that's not necessarily bad thing, but sometimes it just is ! From the old days, I knew that Oracle 8i's CBO (with default settings) is quite fond of full tablescans and parallel query. I had a lot of migrations from 8.0 to 8i were performance went bad just because of that. Now we're not interested in PQ in an OLTP, so we disabled it.
Then there's Oracle 8i favor for full tablescans. The CBO didn't have any CPU costing back then, so one of the things to do was to set optimizer_index_caching and optimizer_index_cost_adj. There have been a lot of discussing of these parameters in the past, but my experience (in 8i and on OLTP db) is that about 95% of all queries benefit and 5% of the queries will perform worse. The main problem with lowering optimizer_index_cost_adj are rounding errors in the CBO's calculation costs. This can lead to the CBO picking an index based on it's alphabetical order.
Simply explained - let's assume the cost of using index A is 9 and the cost of using index B is 5. If optimizer_index_cost_adj is set to 10, then the cost is divided by 10. So the cost of index A will be 0.9 and index B will be 0.5. Or when rounding kicks in, the cost of both A & B will be 1. And the optimizer will choose the index based on the alphabetical order of it's name. (disclaimer - simple explanation - read Jonathan Lewis' book on Oracle CBO )
<!--break-->
As expected - disabling PQ and setting the optimizer_index_x parameters - dramatically improved performance. So now I could focus on individual statements still going slow. And then I hit what I call bad database design.
Some tables were using the CHAR datatype … nothing wrong with that if you like wasting space. But the CHAR datatype was also being used for storing NUMBERs !!! The CHAR columns were indexed but that doesn't help much using where clauses like 'where <char column>=number'.
Let's setup a small testcase :
SQL> create table duh ( n number, c char(30) );
Table created.
SQL> create index i_duh_n on duh(n) ;
Index created.
SQL> create index i_duh_c on duh(c) ;
Index created.
SQL> insert into duh (n,c) select obj#,obj# from sys.obj$ ;
12201 rows created.
SQL> commit ;
Commit complete.
Querying the number column N will result in an index range scan :
SQL> select * from duh where n=2 ;
N C
--------- ------------------------------
2 2
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=2 Bytes=76)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'DUH' (Cost=2 Card=2 Bytes=76)
2 1 INDEX (RANGE SCAN) OF 'I_DUH_N' (NON-UNIQUE) (Cost=1 Car d=2)
No problem here - but when we query the CHAR column C as shown below, the optimizer will perform an implicit cast, and thus no index can be used.
SQL> select * from duh where c=2 ;
N C
--------- ------------------------------
2 2
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=44 Card=123 Bytes=4674)
1 0 TABLE ACCESS (FULL) OF 'DUH' (Cost=44 Card=123 Bytes=4674)
All problematic queries were like the ones above. Sometimes you saw a little variation like this :
SQL> select * from duh where trim(c)='2' ;
N C
--------- ------------------------------
2 2
This means the developers knew something was fishy about using CHAR as a NUMBER ;-)
Anyway - there's a quick fix for this - and it's called function based indexes (FBI). They were introduced in Oracle 8i, so, two thumbs up, I could use them. Well, I could use them after setting QUERY_REWRITE_ENABLED=TRUE and QUERY_REWRITE_INTEGRITY=TRUSTED
But I had already done this as I had seen a lot of 'WHERE UPPER(x)=UPPER(y)' type of where clauses - a developer's favorite performance killer !
So once I identified all the queries that were full table scanning these 'CHAR as NUMBER' tables, I could do the following with it :
SQL> create index i_duh_c_fbi on duh(to_number(c)) ;
Index created.
SQL> select * from duh where c=2 ;
N C
--------- ------------------------------
2 2
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=2 Bytes=76)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'DUH' (Cost=2 Card=2 Bytes=76)
2 1 INDEX (RANGE SCAN) OF 'I_DUH_C_FBI' (NON-UNIQUE) (Cost=1 Card=2)
BONUS! If you know that those tables were around 60 to 80Mb each and those queries were executed quite frequently, it doesn't take a genius to see what that does for IO and performance in general. The SAN could now cope with the number of IO's thrown to it. (see the hockey stick model)
There was, however, one other thing. These tables were also queried from remote databases, mainly one XE database running the web part of the application (APEX).
After the FBI's were implemented, some parts of the web app broke down. Looking at the user dumps, we found out that the remote server process crashed each time a table (or synonym) with an fbi on it was queried. Basically we had an ORA-07445 on the XE database session each time a remote query was launched on the fbi tables :
A brief search on Metalink clarified that this has something todo with 'hidden columns'.
Now ... I could only reproduce the ORA-07445 when the execution plan on the remote DB used the function based index.
So what happens if you create a function based index ? IS this really a kind of index or did the oracle kernel developers added some special trickery to the table ? Or does a function based index add a column to the table behind your back, materializing the result of the function, and indexes that ?
Let's look at the columns of table DUH without function based indexes : SQL> select name,col#,intcol#,segcol#,property from sys.col$ where obj# = ( select obj# from sys.obj$ where name = upper('DUH') and type# = 2);
NAME COL# INTCOL# SEGCOL# PROPERTY
------------------------------ --------- --------- --------- ---------
N 1 1 1 0
C 2 2 2 0
And let's dump a datablock of DUH (without fbi) to trace :
Now lets do the same after we've created a function based index on 'to_number(c)' : SQL> select name,col#,intcol#,segcol#,property from sys.col$ where obj# = ( select obj# from sys.obj$ where name = upper('DUH') and type# = 2);
NAME COL# INTCOL# SEGCOL# PROPERTY
------------------------------ --------- --------- --------- ---------
N 1 1 1 0
C 2 2 2 0 SYS_NC00003$ 0 3 0 327976
Hmm - a system generated column !
Property 327976 or 0x50128=0x40000+0x10000+0x00100+0x00020+0x00008
means
0x40000 = virtual column is NLS dependent
0x10000 = virtual column
0x00100 = system-generated column
0x00020 = hidden column
0x00008 = virtual column
So we have a hidden, system generated, virtual column.
COL# is zero - this is consistent with the column being hidden.
And SEGCOL# is also zero - which tells us that the column is not materialized within the table's blocks and that's consistent with the column being virtual.
You can the property values from sql.bsq in $OH/rdbms/admin : property number not null, /* column properties (bit flags): */
/* 0x0001 = 1 = ADT attribute column */
/* 0x0002 = 2 = OID column */
/* 0x0004 = 4 = nested table column */
/* 0x0008 = 8 = virtual column */
/* 0x0010 = 16 = nested table's SETID$ column */
/* 0x0020 = 32 = hidden column */
/* 0x0040 = 64 = primary-key based OID column */
/* 0x0080 = 128 = column is stored in a lob */
/* 0x0100 = 256 = system-generated column */
/* 0x0200 = 512 = rowinfo column of typed table/view */
/* 0x0400 = 1024 = nested table columns setid */
/* 0x0800 = 2048 = column not insertable */
/* 0x1000 = 4096 = column not updatable */
/* 0x2000 = 8192 = column not deletable */
/* 0x4000 = 16384 = dropped column */
/* 0x8000 = 32768 = unused column - data still in row */
/* 0x00010000 = 65536 = virtual column */
/* 0x00020000 = 131072 = place DESCEND operator on top */
/* 0x00040000 = 262144 = virtual column is NLS dependent */
/* 0x00080000 = 524288 = ref column (present as oid col) */
/* 0x00100000 = 1048576 = hidden snapshot base table column */
/* 0x00200000 = 2097152 = attribute column of a user-defined ref */
/* 0x00400000 = 4194304 = export hidden column,RLS on hidden col */
/* 0x00800000 = 8388608 = string column measured in characters */
/* 0x01000000 = 16777216 = virtual column expression specified */
/* 0x02000000 = 33554432 = typeid column */
/* 0x04000000 = 67108864 = Column is encrypted */
/* 0x20000000 = 536870912 = Column is encrypted without salt */
Let's double check the column hasn't materialized by dumping a block.
COL# is non-zero - so it is visable. Clearly it's not system generated, but it isn't materialised because SEGCOL# is zero.
Now for fun - let's create an index on the virtual column V.
SQL> create index i_duh11g_v on duh11g( v ) ;
Index created.
And now dump a leafblock of the index :
Block header dump: 0x004159db
Object id on Block? Y
seg/obj: 0x11587 csc: 0x00.ba00b itc: 2 flg: - typ: 2 - INDEX
fsl: 0 fnx: 0x0 ver: 0x01
Itl Xid Uba Flag Lck Scn/Fsc
0x01 0x0006.001.000002ed 0x00c0028d.01f3.01 -B-- 1 fsc 0x0000.00000000
0x02 0x0009.002.000002b7 0x00c0168b.0216.04 ---- 540 fsc 0x0000.00000000
Leaf block dump
===============
header address 682344540=0x28abc05c
kdxcolev 0
KDXCOLEV Flags = - - -
kdxcolok 1
kdxcoopc 0x87: opcode=7: iot flags=--- is converted=Y
kdxconco 2
kdxcosdc 1
kdxconro 540
kdxcofbo 1116=0x45c
kdxcofeo 1116=0x45c
kdxcoavs 0
kdxlespl 0
kdxlende 0
kdxlenxt 4282844=0x4159dc
kdxleprv 0=0x0
kdxledsz 0
kdxlebksz 8032
row#0[1116] flag: ----S-, lock: 2, len=12
col 0; len 2; (2): c1 02
col 1; len 6; (6): 00 41 59 c2 00 00
row#1[1128] flag: ----S-, lock: 2, len=12
col 0; len 2; (2): c1 03
col 1; len 6; (6): 00 41 59 c2 00 01
...
row#539[8019] flag: ----S-, lock: 2, len=13
col 0; len 3; (3): c2 06 29
col 1; len 6; (6): 00 41 59 c4 00 ae
----- end of leaf block dump -----
What we got is that a normal index on a virtual column in 11g is the same as a function based index in 8i.
And if you use an fbi in 8i you have actually created a hidden virtual column. It doesn't add a physical column to your table, which of course would be very io intensive and could have let to row migration (and possible chaining).
So to get back to the bug - the work around is to created a view on the table in the remote database and then query the view as opposed to querying the actual table or synonym.
This bug is also fixed in 9.2.0.6 and 10g.... but then again ... we had to work with 8.1.7.4
]]>
I was called in last week for a performance issue at a customer site.
Let's set scene a bit …
They had moved their main production OLTP database from AIX to Windows and had been struggling with performance for a couple of weeks. Hardware-wise, they went from expensive but old Bull hardware to a Dell box (with 8 cores). And they went from 4Gb to 16Gb RAM. So you would expect the database would fly on this new hardware.
Now - this database was/is pulling around a 1000 concurrent users…. The first thing they ran into was the process' addressable memory space on 32bit windows, being 2Gb. This of course led to a lot of ORA-04030 and angry users. Actually, Anjo recently made a blogentry about it.
So actions were made to optimize Oracle for windows, which basically consists of putting the buffercache above the 2Gb limit - anyway metalink note 231159.1 explains all of this, but still, as this is an Oracle 8.1.7.4 database handling a thousand users, all other memory (remember sort_area_size, hash_area_size etc) related parameters were set to a minimum to avoid ORA-4030's.
The result of this memory balancing operation was, that performance was below average. And I was send in to have a look at it. I must admit - I hadn't tuned a whole environment for a while, but I had done a lot of 8.0 and 7.3 migrations to 8i back in the day, and knew pretty much what the problems of the 8i CBO were. Oh - why are the still running 8i … well the database application is to be decommissioned this year and so they didn't bother to move up to the latest Oracle version. The fact that they wanted to invest in getting performance up, proves power users were about to lynch the IT manager
Anyway - 5minutes into the meeting with IT, a second consultant from a rivaling company walks in. That's how bad performance was. I don't mind competition - in fact I didn't see it as competition. The guy was more than reasonable and during the day we acted as each-others soundboard.
The first thing I noticed were the bad response times on the disks and the large amount of physical IO the database was doing. The database running on a Dell SAN…. I didn't even know Dell produced SANs !!! :-p But I knew one thing - this SAN was sh*te. It used 8 disks in RAID5 (coming from 4 disks) and I reckon the write cache must have been quite small. In fact at first I thought the SAN cache must have been disabled - something I had encountered multiple times when the battery of the cache dies, the cache on the SAN is disabled to avoid corruption in case of power-failure.
Clearly, this DB was doing too much IO for the SAN… way past it's critical point in the hockey stick chart see also Anjo's paper on this - oh and we couldn't increase any memory settings within the DB of course.
So the only option left, was hunting down individual sql statements and tune them, so the overall IO would decrease. The impact of sql tuning can be *huge* - especially with a 1000 oltp users. Just imaging a sql that does 5000 physical IO's that is run by these 1000 users every minute (5x10^6 IO/min) - if we can tune it to use only 20 physical IO's (20x10^3 IO/min) than that would have a dramatic effect on overall performance.
I noticed that there were a lot of full tablescans … that's not necessarily bad thing, but sometimes it just is ! From the old days, I knew that Oracle 8i's CBO (with default settings) is quite fond of full tablescans and parallel query. I had a lot of migrations from 8.0 to 8i were performance went bad just because of that. Now we're not interested in PQ in an OLTP, so we disabled it.
Then there's Oracle 8i favor for full tablescans. The CBO didn't have any CPU costing back then, so one of the things to do was to set optimizer_index_caching and optimizer_index_cost_adj. There have been a lot of discussing of these parameters in the past, but my experience (in 8i and on OLTP db) is that about 95% of all queries benefit and 5% of the queries will perform worse. The main problem with lowering optimizer_index_cost_adj are rounding errors in the CBO's calculation costs. This can lead to the CBO picking an index based on it's alphabetical order.
Simply explained - let's assume the cost of using index A is 9 and the cost of using index B is 5. If optimizer_index_cost_adj is set to 10, then the cost is divided by 10. So the cost of index A will be 0.9 and index B will be 0.5. Or when rounding kicks in, the cost of both A & B will be 1. And the optimizer will choose the index based on the alphabetical order of it's name. (disclaimer - simple explanation - read Jonathan Lewis' book on Oracle CBO )
<!--break-->
As expected - disabling PQ and setting the optimizer_index_x parameters - dramatically improved performance. So now I could focus on individual statements still going slow. And then I hit what I call bad database design.
Some tables were using the CHAR datatype … nothing wrong with that if you like wasting space. But the CHAR datatype was also being used for storing NUMBERs !!! The CHAR columns were indexed but that doesn't help much using where clauses like 'where <char column>=number'.
Let's setup a small testcase :
SQL> create table duh ( n number, c char(30) );
Table created.
SQL> create index i_duh_n on duh(n) ;
Index created.
SQL> create index i_duh_c on duh(c) ;
Index created.
SQL> insert into duh (n,c) select obj#,obj# from sys.obj$ ;
12201 rows created.
SQL> commit ;
Commit complete.
Querying the number column N will result in an index range scan :
SQL> select * from duh where n=2 ;
N C
--------- ------------------------------
2 2
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=2 Bytes=76)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'DUH' (Cost=2 Card=2 Bytes=76)
2 1 INDEX (RANGE SCAN) OF 'I_DUH_N' (NON-UNIQUE) (Cost=1 Car d=2)
No problem here - but when we query the CHAR column C as shown below, the optimizer will perform an implicit cast, and thus no index can be used.
SQL> select * from duh where c=2 ;
N C
--------- ------------------------------
2 2
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=44 Card=123 Bytes=4674)
1 0 TABLE ACCESS (FULL) OF 'DUH' (Cost=44 Card=123 Bytes=4674)
All problematic queries were like the ones above. Sometimes you saw a little variation like this :
SQL> select * from duh where trim(c)='2' ;
N C
--------- ------------------------------
2 2
This means the developers knew something was fishy about using CHAR as a NUMBER ;-)
Anyway - there's a quick fix for this - and it's called function based indexes (FBI). They were introduced in Oracle 8i, so, two thumbs up, I could use them. Well, I could use them after setting QUERY_REWRITE_ENABLED=TRUE and QUERY_REWRITE_INTEGRITY=TRUSTED
But I had already done this as I had seen a lot of 'WHERE UPPER(x)=UPPER(y)' type of where clauses - a developer's favorite performance killer !
So once I identified all the queries that were full table scanning these 'CHAR as NUMBER' tables, I could do the following with it :
SQL> create index i_duh_c_fbi on duh(to_number(c)) ;
Index created.
SQL> select * from duh where c=2 ;
N C
--------- ------------------------------
2 2
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=2 Bytes=76)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'DUH' (Cost=2 Card=2 Bytes=76)
2 1 INDEX (RANGE SCAN) OF 'I_DUH_C_FBI' (NON-UNIQUE) (Cost=1 Card=2)
BONUS! If you know that those tables were around 60 to 80Mb each and those queries were executed quite frequently, it doesn't take a genius to see what that does for IO and performance in general. The SAN could now cope with the number of IO's thrown to it. (see the hockey stick model)
There was, however, one other thing. These tables were also queried from remote databases, mainly one XE database running the web part of the application (APEX).
After the FBI's were implemented, some parts of the web app broke down. Looking at the user dumps, we found out that the remote server process crashed each time a table (or synonym) with an fbi on it was queried. Basically we had an ORA-07445 on the XE database session each time a remote query was launched on the fbi tables :
A brief search on Metalink clarified that this has something todo with 'hidden columns'.
Now ... I could only reproduce the ORA-07445 when the execution plan on the remote DB used the function based index.
So what happens if you create a function based index ? IS this really a kind of index or did the oracle kernel developers added some special trickery to the table ? Or does a function based index add a column to the table behind your back, materializing the result of the function, and indexes that ?
Let's look at the columns of table DUH without function based indexes : SQL> select name,col#,intcol#,segcol#,property from sys.col$ where obj# = ( select obj# from sys.obj$ where name = upper('DUH') and type# = 2);
NAME COL# INTCOL# SEGCOL# PROPERTY
------------------------------ --------- --------- --------- ---------
N 1 1 1 0
C 2 2 2 0
And let's dump a datablock of DUH (without fbi) to trace :
Now lets do the same after we've created a function based index on 'to_number(c)' : SQL> select name,col#,intcol#,segcol#,property from sys.col$ where obj# = ( select obj# from sys.obj$ where name = upper('DUH') and type# = 2);
NAME COL# INTCOL# SEGCOL# PROPERTY
------------------------------ --------- --------- --------- ---------
N 1 1 1 0
C 2 2 2 0 SYS_NC00003$ 0 3 0 327976
Hmm - a system generated column !
Property 327976 or 0x50128=0x40000+0x10000+0x00100+0x00020+0x00008
means
0x40000 = virtual column is NLS dependent
0x10000 = virtual column
0x00100 = system-generated column
0x00020 = hidden column
0x00008 = virtual column
So we have a hidden, system generated, virtual column.
COL# is zero - this is consistent with the column being hidden.
And SEGCOL# is also zero - which tells us that the column is not materialized within the table's blocks and that's consistent with the column being virtual.
You can the property values from sql.bsq in $OH/rdbms/admin : property number not null, /* column properties (bit flags): */
/* 0x0001 = 1 = ADT attribute column */
/* 0x0002 = 2 = OID column */
/* 0x0004 = 4 = nested table column */
/* 0x0008 = 8 = virtual column */
/* 0x0010 = 16 = nested table's SETID$ column */
/* 0x0020 = 32 = hidden column */
/* 0x0040 = 64 = primary-key based OID column */
/* 0x0080 = 128 = column is stored in a lob */
/* 0x0100 = 256 = system-generated column */
/* 0x0200 = 512 = rowinfo column of typed table/view */
/* 0x0400 = 1024 = nested table columns setid */
/* 0x0800 = 2048 = column not insertable */
/* 0x1000 = 4096 = column not updatable */
/* 0x2000 = 8192 = column not deletable */
/* 0x4000 = 16384 = dropped column */
/* 0x8000 = 32768 = unused column - data still in row */
/* 0x00010000 = 65536 = virtual column */
/* 0x00020000 = 131072 = place DESCEND operator on top */
/* 0x00040000 = 262144 = virtual column is NLS dependent */
/* 0x00080000 = 524288 = ref column (present as oid col) */
/* 0x00100000 = 1048576 = hidden snapshot base table column */
/* 0x00200000 = 2097152 = attribute column of a user-defined ref */
/* 0x00400000 = 4194304 = export hidden column,RLS on hidden col */
/* 0x00800000 = 8388608 = string column measured in characters */
/* 0x01000000 = 16777216 = virtual column expression specified */
/* 0x02000000 = 33554432 = typeid column */
/* 0x04000000 = 67108864 = Column is encrypted */
/* 0x20000000 = 536870912 = Column is encrypted without salt */
Let's double check the column hasn't materialized by dumping a block.
COL# is non-zero - so it is visable. Clearly it's not system generated, but it isn't materialised because SEGCOL# is zero.
Now for fun - let's create an index on the virtual column V.
SQL> create index i_duh11g_v on duh11g( v ) ;
Index created.
And now dump a leafblock of the index :
Block header dump: 0x004159db
Object id on Block? Y
seg/obj: 0x11587 csc: 0x00.ba00b itc: 2 flg: - typ: 2 - INDEX
fsl: 0 fnx: 0x0 ver: 0x01
Itl Xid Uba Flag Lck Scn/Fsc
0x01 0x0006.001.000002ed 0x00c0028d.01f3.01 -B-- 1 fsc 0x0000.00000000
0x02 0x0009.002.000002b7 0x00c0168b.0216.04 ---- 540 fsc 0x0000.00000000
Leaf block dump
===============
header address 682344540=0x28abc05c
kdxcolev 0
KDXCOLEV Flags = - - -
kdxcolok 1
kdxcoopc 0x87: opcode=7: iot flags=--- is converted=Y
kdxconco 2
kdxcosdc 1
kdxconro 540
kdxcofbo 1116=0x45c
kdxcofeo 1116=0x45c
kdxcoavs 0
kdxlespl 0
kdxlende 0
kdxlenxt 4282844=0x4159dc
kdxleprv 0=0x0
kdxledsz 0
kdxlebksz 8032
row#0[1116] flag: ----S-, lock: 2, len=12
col 0; len 2; (2): c1 02
col 1; len 6; (6): 00 41 59 c2 00 00
row#1[1128] flag: ----S-, lock: 2, len=12
col 0; len 2; (2): c1 03
col 1; len 6; (6): 00 41 59 c2 00 01
...
row#539[8019] flag: ----S-, lock: 2, len=13
col 0; len 3; (3): c2 06 29
col 1; len 6; (6): 00 41 59 c4 00 ae
----- end of leaf block dump -----
What we got is that a normal index on a virtual column in 11g is the same as a function based index in 8i.
And if you use an fbi in 8i you have actually created a hidden virtual column. It doesn't add a physical column to your table, which of course would be very io intensive and could have let to row migration (and possible chaining).
So to get back to the bug - the work around is to created a view on the table in the remote database and then query the view as opposed to querying the actual table or synonym.
This bug is also fixed in 9.2.0.6 and 10g.... but then again ... we had to work with 8.1.7.4
]]>AXI at Ostend new year's plungehttp://www.ora600.be/axi-at-new-year-plunge-ostend2009-01-04T11:03:37+01:002009-01-04T11:09:17+01:00kurtvm
It's been freezing here for the last two weeks … the hardest and longest since 1993 (where are those green guys now, telling me about cutting my engine power because of global warming ?).
Anyway - the company I work for - AXI, an Oracle partner in Belgium and The Netherlands - decided to take part in the new year's plunge in Ostend.
Yes - you read it right - not a plunge in the sea at sunny Cape Town, or Sidney Australia - but in freezing Ostend.
Of course this is not healthy at all - many doctors warned about it. "The freezing cold is bad for heart and blood vessels. A big change in pressure or big changes in temperature can result in sudden heart failure, while it is dangerous for the lungs too."
Once that hit the news - and once a similar event was canceled in Holland - I knew the Belgian event would hit a larger crowd than ever, because idiots as we are, we want to prove everybody wrong. And 6500 idiots actually went into the sea.
Me and the wife, both tactical thinkers, decided it was more opportune to provide for the catering and marketing material. And as news crews would be on site, we decided to bring along a 2 by 3 meters ( or that's 6.5 by 10 feet ) flag. We had a small logistical problem as we needed to find poles to keep it stretched and … not entirely unimportant, could fit in our car. So we ended up buying some retractable poles - originally intended for fishing nets - from a fishing shop
I must say - the company name was extremely visible even in between a crowd of more than 6000
Our first mission was accomplished.
It is an unwritten rule that if you do something crazy and irresponsive as diving into 3°C sea water at freezing temperature, that you should drink some alcohol afterwards to … warm you up. It's the logical thing to do (if you're going for a heart attack, you should go all the way) And the weapon of choice around this time of the year is … jenever
Because me and the wife were providing for the catering, I was dragging along a backpack with 8 bottles of jenever… this meant I had … a god-like status - I was Bacchus himself
Anyway - here are some snapshots
Marketing at it's finest
The smartest guy of the group - Bacchus, god of jenever and marketing materials - all warm and fully clothed !
]]>
It's been freezing here for the last two weeks … the hardest and longest since 1993 (where are those green guys now, telling me about cutting my engine power because of global warming ?).
Anyway - the company I work for - AXI, an Oracle partner in Belgium and The Netherlands - decided to take part in the new year's plunge in Ostend.
Yes - you read it right - not a plunge in the sea at sunny Cape Town, or Sidney Australia - but in freezing Ostend.
Of course this is not healthy at all - many doctors warned about it. "The freezing cold is bad for heart and blood vessels. A big change in pressure or big changes in temperature can result in sudden heart failure, while it is dangerous for the lungs too."
Once that hit the news - and once a similar event was canceled in Holland - I knew the Belgian event would hit a larger crowd than ever, because idiots as we are, we want to prove everybody wrong. And 6500 idiots actually went into the sea.
Me and the wife, both tactical thinkers, decided it was more opportune to provide for the catering and marketing material. And as news crews would be on site, we decided to bring along a 2 by 3 meters ( or that's 6.5 by 10 feet ) flag. We had a small logistical problem as we needed to find poles to keep it stretched and … not entirely unimportant, could fit in our car. So we ended up buying some retractable poles - originally intended for fishing nets - from a fishing shop
I must say - the company name was extremely visible even in between a crowd of more than 6000
Our first mission was accomplished.
It is an unwritten rule that if you do something crazy and irresponsive as diving into 3°C sea water at freezing temperature, that you should drink some alcohol afterwards to … warm you up. It's the logical thing to do (if you're going for a heart attack, you should go all the way) And the weapon of choice around this time of the year is … jenever
Because me and the wife were providing for the catering, I was dragging along a backpack with 8 bottles of jenever… this meant I had … a god-like status - I was Bacchus himself
Anyway - here are some snapshots
Marketing at it's finest
The smartest guy of the group - Bacchus, god of jenever and marketing materials - all warm and fully clothed !
]]>Hunting down partitioned IOT objectid'shttp://www.ora600.be/partitioned-iot-objectid2008-12-30T10:10:19+01:002008-12-30T10:10:19+01:00kurtvm
I'm so glad you can't sub-partition an indexed organized table (IOT). (no we do not want this feature in 12g)
I was looking into my code for unloading partitioned IOT's.
Unloading IOT's (with or without overflow segments) is easy. It's keeping track of all the meta-data that's starting to get difficult.
DUDE allows you to unload tables as follows :
Table by table based on owner and tablename or objectid
Shema by schema - so based on schema/owner name
Tablespace by tablespace - and this is based on the data objectid of the table
You basically can define three different types of object id's
There's the objectid itself, which defines the object uniquely in most if not all base dictionary table
Then there's the data objectid (or to use an expensive word 'data layer objectid' - which is the objectid stored within the block itself. Mostly it equals objectid, but if you for example move or truncate a table it'll change. A data objectid can be NULL - for example the table part of a partitioned table does not have a physical segment - it's the partitions that actually contain the data
And then there's the base objectid - which is the objectid of the base table of an object … or a pointer to another object to put it simple.
So - if you're unloading a complete tablespace, you are going through the data objectid's found in the tablespace. And using those physical objectid's you have to move your way up to the table objectid using the metadata in the base dictionary tables.
Why ? Because you need the table definition (name, columns, datatypes etc). This is quite simple tho.
But wait, sometimes it can get messy. Let's imaging a partitioned IOT using overflow segments with <n> partitions.
What you get is the following meta data spread around several base dictionary tables :
1 table : objectid is <t>, data objectid is null, base objectid is <o>
<n> table partitions : objectid is <t+n>, data objectid is null, base objectid is <t>
1 index : objectid is <i>, data objectid is null, base objectid is <t>
<n> index partitions : objectid is <i+1>, data objectid is <i+1> initially, base objectid is <i>
1 table (overflow) : objectid is <o>, data objectid is null, base objectid is <t>
<n> table partitions (overflow) : objectid is <o+1>, data objectid is <o+1> initially, base objectid is <o>
As you can see - the table is linked to the overflow table and vice versa using a base objectid.
The table partitions are linked to the table, the index is connected to the table, the index partitions are hooked up to the index, and the overflow table partitions are linked to the overflow table… all using the base objectid's. And only two types of objects have actual physical segments - the index partitions and the overflow table partitions. So if DUDE hits a data objectid of an IOT index partition it can traverse up to the table object and get the definitions.
Pfew … the whole thing made me think about the Front 242 song 'Headhunter' - '1 - you lock the target - 2 - you bait the line - 3 - you slowly spread the net - 4 - you catch the … euh objectid ?'
I'm sure there are a couple of Oracle nerds out there that can appreciate this clip for 1988 !
]]>
I'm so glad you can't sub-partition an indexed organized table (IOT). (no we do not want this feature in 12g)
I was looking into my code for unloading partitioned IOT's.
Unloading IOT's (with or without overflow segments) is easy. It's keeping track of all the meta-data that's starting to get difficult.
DUDE allows you to unload tables as follows :
Table by table based on owner and tablename or objectid
Shema by schema - so based on schema/owner name
Tablespace by tablespace - and this is based on the data objectid of the table
You basically can define three different types of object id's
There's the objectid itself, which defines the object uniquely in most if not all base dictionary table
Then there's the data objectid (or to use an expensive word 'data layer objectid' - which is the objectid stored within the block itself. Mostly it equals objectid, but if you for example move or truncate a table it'll change. A data objectid can be NULL - for example the table part of a partitioned table does not have a physical segment - it's the partitions that actually contain the data
And then there's the base objectid - which is the objectid of the base table of an object … or a pointer to another object to put it simple.
So - if you're unloading a complete tablespace, you are going through the data objectid's found in the tablespace. And using those physical objectid's you have to move your way up to the table objectid using the metadata in the base dictionary tables.
Why ? Because you need the table definition (name, columns, datatypes etc). This is quite simple tho.
But wait, sometimes it can get messy. Let's imaging a partitioned IOT using overflow segments with <n> partitions.
What you get is the following meta data spread around several base dictionary tables :
1 table : objectid is <t>, data objectid is null, base objectid is <o>
<n> table partitions : objectid is <t+n>, data objectid is null, base objectid is <t>
1 index : objectid is <i>, data objectid is null, base objectid is <t>
<n> index partitions : objectid is <i+1>, data objectid is <i+1> initially, base objectid is <i>
1 table (overflow) : objectid is <o>, data objectid is null, base objectid is <t>
<n> table partitions (overflow) : objectid is <o+1>, data objectid is <o+1> initially, base objectid is <o>
As you can see - the table is linked to the overflow table and vice versa using a base objectid.
The table partitions are linked to the table, the index is connected to the table, the index partitions are hooked up to the index, and the overflow table partitions are linked to the overflow table… all using the base objectid's. And only two types of objects have actual physical segments - the index partitions and the overflow table partitions. So if DUDE hits a data objectid of an IOT index partition it can traverse up to the table object and get the definitions.
Pfew … the whole thing made me think about the Front 242 song 'Headhunter' - '1 - you lock the target - 2 - you bait the line - 3 - you slowly spread the net - 4 - you catch the … euh objectid ?'
I'm sure there are a couple of Oracle nerds out there that can appreciate this clip for 1988 !
]]>I love slidesharehttp://www.ora600.be/extending-the-oracle-sso-server-with-saml-and-pki2008-12-29T14:08:21+01:002008-12-29T14:11:44+01:00kurtvm
Some idea's are so cool I'm just jealous I didn't think of them myself. With all the crappy, time consuming and totally useless social network sites (linkedin being the exception) it's refreshing to see something like slideshare.com.
The idea is pretty simple - share your slides ! In fact, now that I think of it - it's basically like youtube - but with slides - or presentations.
Just create an account - upload your presentations and slideshare will convert your pdf's or ppt's or whatever format into a flash driven presentation. Which at that point, you'll be able to embed them into your website.
Just for fun I uploaded the presentations I did for UKOUG2007 and UKOUG2008. The 2007 one got selected by the slideshare editorial team as a showcase on their technology page. Way cool!
Your presentation Extending the Oracle SSO is currently being showcased on the 'Technology' page by our editorial team.
It's likely to be there for the next 16-20 hours...
Cheers,
- the SlideShare team
I'm pretty sure their editorial team is a couple of uber-geeks as the selected presentation 'Extending the Oracle Single-Sign-On server' is … well … kind of geeky … and I even dumb'ed it down a bit. It's all about PKI, certificates, authentication, reverse proxies, apache and … fitting Oracle's SSO server somewhere in between
It's also loaded with PowerPoint's custom animations, so you actually need to download it and run it in PowerPoint to get an idea of what I'm aiming for … so the editorial team must have been geeky enough to download it and let all the animations spin their heads around.
The content itself is based on the work I've been doing at a customer's site - an insurance company - that had very specific requirements regarding their SSO architecture. They had made Oracle their choice of platform as they had been using the Oracle RDBMS for about 10years (I was the one who set it up back then). So it was only natural that when they wanted to 'web-enable' their back-office applications, and give their - and third party - brokers access to some of their back-office applications, they chose Oracle's Internet Application Server.
As all their business logic was built within PLSQL - the fat database model or database centric approach as Toon Koppelaars calls it - it was quite easy to develop a web front-end for it. The thing that they struggled the most with, was integrating Oracle single sign on with their fancy and expensive vpn/ssl boxes. Not only that but, they needed to integrate Oracle SSO with a federated authentication infrastructure, so that third party brokers, who were already authenticated through a third party identity provider could automatically logon to their applications, mapping third party identities to their identities.
And we're only halfway there - they also wanted to run their own certificate authority (thank god for Oracle CA … are we the only one actually using it in production ? ). Using their own CA they wanted to generate client certificates so that internet brokers could authenticate themselves with their own client certificate. But they also had the choice of using their Belgian passport to authenticate themselves against Oracle SSO. And because the insurance holding has multiple companies, the logon screens needed multiple look-and-feels … oh and multi-language support of course.
So for the last couple of years I have been busy pulling Oracle's SSO server inside-out, integrating it with a juniper 4000 VPN/SSL series - which we threw out in favor of an apache2 based SSL terminator and reverse proxy. The juniper box was just holding us back. I did have to write some custom apache filters for Oracle Portal. I think Oracle Portal generates some of the weirdest html I've ever seen… and mod_proxy_html wasn't up to the job for that
I've also integrated SAMLv2 with Oracle SSO. I had to write a java based SAML proxy to get it done, because the SAML token needed to be transferred over a HTTPs connection using a client certificate authentication - but it works like a charm. This was only done this year - so it's only slightly mentioned in the presentation. We had a kind of DIY federated authentication before that, as the third party was not finished implementing a SAML IP themselves.
So I wrote various software packages that tied it all together, like :
User admin packages : delegate user administration - this is a pure PLSQL API. This is our own version of OIDDAS on steroids. It can
create/delete users
change user passwords - force password changes - enforce minimum password requirements
change/get user LDAP attributes
do identity bootstrapping for federated authentication
request/revoke client certificates for users - update the CRL
restrict access to applications depending on whether they are on the LAN, internet or private network (third party)
synchronize Oracle SSO users with users of a MS IIS/ASP/Sybase based application
login proxies and plugin's for SSO, which enables us to
login based on client certificates/password for internet users
login based on eID certificate for internet users
login based on SAMLv2 token for third party brokers
login based on a DIY federated authentication for third party brokers
login based on username/password for LAN back-office user
login via Oracle SSO to a Microsoft IIS/ASP/Sybase application (oh yes - we did it!)
OS/browser detection - quite handy for VISTA and webforms, so we can switch a webforms config transparently if we detect vista. For example - the url /forms/frmservlet?config=app is rewritten to /forms/frmservlet?config=va_app transparently when vista is detected
User-based Oracle Webforms config - we can also change Webforms configs on a user per user basis. This is quite handy for debugging or if one specific user has troubles. For example /forms/frmservlet?config=app is rewritten to /forms/frmservlet?config=d_app for certain users. We do this by de-obfuscating the sitetoken,which will give us the URL, lookup the SSO user and see if we need to change the URL, rewrite the URL and then obfuscate the URL back to a sitetoken. Once the sitetoken reaches mod_osso it will redirect to the modified URL.
It does lots more things like multi-language detection, different look-and-feel depending on which apache virtualhost your coming from etc
Certificate manager - we use the Oracle CA (OCA) quite extensively - but the user and admin screen are … let's say … not so user friendly … and you can't adjust the look and feel just the way you like. So I wrote what I call a certificate manager.
The certificate manager uses openssl to create a private key and certificate request on the infrastructure server. This, as opposed to using activeX, javascript and the browser's wallet. The certificate manager picks up requests from our user admin package.
The certificate manager then submits the request to the OCA.
Someone from the company then approves the request (also through the certificate manager)
And then the certificate is packaged in a password protected PKCS12 wallet with the private key and is shipped to the broker in a secure way (which I won't go into details here)
The certificate manager uses a bunch of techniques and also has a PLSQL wrapper as an API - so there's no need to use the actual OCA webpages. This means we can easily delegate administration. For example - if an employee of a broker office leaves or gets fired, the head of the office can revoke the client certificate through his application, which will in turn update the CRL and disable the user in OID. Of course - approval of client certificates is always done centrally by the head office.
Anyway - that's in a nutshell what the presentation is about. Never mind all the DBA stuff and java programming … working with these technologies was awesome … and it'll help me a grea lot in my DUDE work when I finally finish support for unloading data encrypted using Transparent Data Encryption (TDE)
Oh - here's the presentation embedded ... but you'll need to download it and run it powerpoint to get the animations running !
]]>
Some idea's are so cool I'm just jealous I didn't think of them myself. With all the crappy, time consuming and totally useless social network sites (linkedin being the exception) it's refreshing to see something like slideshare.com.
The idea is pretty simple - share your slides ! In fact, now that I think of it - it's basically like youtube - but with slides - or presentations.
Just create an account - upload your presentations and slideshare will convert your pdf's or ppt's or whatever format into a flash driven presentation. Which at that point, you'll be able to embed them into your website.
Just for fun I uploaded the presentations I did for UKOUG2007 and UKOUG2008. The 2007 one got selected by the slideshare editorial team as a showcase on their technology page. Way cool!
Your presentation Extending the Oracle SSO is currently being showcased on the 'Technology' page by our editorial team.
It's likely to be there for the next 16-20 hours...
Cheers,
- the SlideShare team
I'm pretty sure their editorial team is a couple of uber-geeks as the selected presentation 'Extending the Oracle Single-Sign-On server' is … well … kind of geeky … and I even dumb'ed it down a bit. It's all about PKI, certificates, authentication, reverse proxies, apache and … fitting Oracle's SSO server somewhere in between
It's also loaded with PowerPoint's custom animations, so you actually need to download it and run it in PowerPoint to get an idea of what I'm aiming for … so the editorial team must have been geeky enough to download it and let all the animations spin their heads around.
The content itself is based on the work I've been doing at a customer's site - an insurance company - that had very specific requirements regarding their SSO architecture. They had made Oracle their choice of platform as they had been using the Oracle RDBMS for about 10years (I was the one who set it up back then). So it was only natural that when they wanted to 'web-enable' their back-office applications, and give their - and third party - brokers access to some of their back-office applications, they chose Oracle's Internet Application Server.
As all their business logic was built within PLSQL - the fat database model or database centric approach as Toon Koppelaars calls it - it was quite easy to develop a web front-end for it. The thing that they struggled the most with, was integrating Oracle single sign on with their fancy and expensive vpn/ssl boxes. Not only that but, they needed to integrate Oracle SSO with a federated authentication infrastructure, so that third party brokers, who were already authenticated through a third party identity provider could automatically logon to their applications, mapping third party identities to their identities.
And we're only halfway there - they also wanted to run their own certificate authority (thank god for Oracle CA … are we the only one actually using it in production ? ). Using their own CA they wanted to generate client certificates so that internet brokers could authenticate themselves with their own client certificate. But they also had the choice of using their Belgian passport to authenticate themselves against Oracle SSO. And because the insurance holding has multiple companies, the logon screens needed multiple look-and-feels … oh and multi-language support of course.
So for the last couple of years I have been busy pulling Oracle's SSO server inside-out, integrating it with a juniper 4000 VPN/SSL series - which we threw out in favor of an apache2 based SSL terminator and reverse proxy. The juniper box was just holding us back. I did have to write some custom apache filters for Oracle Portal. I think Oracle Portal generates some of the weirdest html I've ever seen… and mod_proxy_html wasn't up to the job for that
I've also integrated SAMLv2 with Oracle SSO. I had to write a java based SAML proxy to get it done, because the SAML token needed to be transferred over a HTTPs connection using a client certificate authentication - but it works like a charm. This was only done this year - so it's only slightly mentioned in the presentation. We had a kind of DIY federated authentication before that, as the third party was not finished implementing a SAML IP themselves.
So I wrote various software packages that tied it all together, like :
User admin packages : delegate user administration - this is a pure PLSQL API. This is our own version of OIDDAS on steroids. It can
create/delete users
change user passwords - force password changes - enforce minimum password requirements
change/get user LDAP attributes
do identity bootstrapping for federated authentication
request/revoke client certificates for users - update the CRL
restrict access to applications depending on whether they are on the LAN, internet or private network (third party)
synchronize Oracle SSO users with users of a MS IIS/ASP/Sybase based application
login proxies and plugin's for SSO, which enables us to
login based on client certificates/password for internet users
login based on eID certificate for internet users
login based on SAMLv2 token for third party brokers
login based on a DIY federated authentication for third party brokers
login based on username/password for LAN back-office user
login via Oracle SSO to a Microsoft IIS/ASP/Sybase application (oh yes - we did it!)
OS/browser detection - quite handy for VISTA and webforms, so we can switch a webforms config transparently if we detect vista. For example - the url /forms/frmservlet?config=app is rewritten to /forms/frmservlet?config=va_app transparently when vista is detected
User-based Oracle Webforms config - we can also change Webforms configs on a user per user basis. This is quite handy for debugging or if one specific user has troubles. For example /forms/frmservlet?config=app is rewritten to /forms/frmservlet?config=d_app for certain users. We do this by de-obfuscating the sitetoken,which will give us the URL, lookup the SSO user and see if we need to change the URL, rewrite the URL and then obfuscate the URL back to a sitetoken. Once the sitetoken reaches mod_osso it will redirect to the modified URL.
It does lots more things like multi-language detection, different look-and-feel depending on which apache virtualhost your coming from etc
Certificate manager - we use the Oracle CA (OCA) quite extensively - but the user and admin screen are … let's say … not so user friendly … and you can't adjust the look and feel just the way you like. So I wrote what I call a certificate manager.
The certificate manager uses openssl to create a private key and certificate request on the infrastructure server. This, as opposed to using activeX, javascript and the browser's wallet. The certificate manager picks up requests from our user admin package.
The certificate manager then submits the request to the OCA.
Someone from the company then approves the request (also through the certificate manager)
And then the certificate is packaged in a password protected PKCS12 wallet with the private key and is shipped to the broker in a secure way (which I won't go into details here)
The certificate manager uses a bunch of techniques and also has a PLSQL wrapper as an API - so there's no need to use the actual OCA webpages. This means we can easily delegate administration. For example - if an employee of a broker office leaves or gets fired, the head of the office can revoke the client certificate through his application, which will in turn update the CRL and disable the user in OID. Of course - approval of client certificates is always done centrally by the head office.
Anyway - that's in a nutshell what the presentation is about. Never mind all the DBA stuff and java programming … working with these technologies was awesome … and it'll help me a grea lot in my DUDE work when I finally finish support for unloading data encrypted using Transparent Data Encryption (TDE)
Oh - here's the presentation embedded ... but you'll need to download it and run it powerpoint to get the animations running !
]]>Taking another blow in the facehttp://www.ora600.be/ukoug-2008-was-a-blast2008-12-16T21:53:06+01:002008-12-17T14:29:25+01:00kurtvm
Well well well - I reckon this month is 'poke the belgian in the face' month !
I knew my last post was going to be trouble ;-) That'll teach me (I'm sure a lot of people are having a laugh now!)
I kind of suspected someone, somewhere in the world would be offended by it. The world has become a small place, and cultures differ.
Hey, I like to eat horse meat stew with my fries and mayonnaise on the side - I know this is completely not done in some parts of the world - some prefer ketchup
In parts of the world it's perfectly legal to carry guns and it's part of their legacy, it's part of their constitution - so I respect that.
What I don't like is my situation being exploited by certain people who are frustrated with the UKOUG.
This certain person - who we will give the fictional name 'Dom' - turns my small incident into a 'vicious attack'… man - I'm lucky to be alive if I read all this. He then goes on into explaining, 'how everybody is against Americans'… he and his wife were verbally threatened when they were once abroad (they were called 'yanks').
Well - what can I say - I'm pretty sure that's not a pleasant situation when you're in a foreign country, I can't deny that. But I actually happen to like Americans as some of my best and dearest friends are yanks ;-) (Which is not an offensive word where I come from - but again cultures differ)
]]>
Well well well - I reckon this month is 'poke the belgian in the face' month !
I knew my last post was going to be trouble ;-) That'll teach me (I'm sure a lot of people are having a laugh now!)
I kind of suspected someone, somewhere in the world would be offended by it. The world has become a small place, and cultures differ.
Hey, I like to eat horse meat stew with my fries and mayonnaise on the side - I know this is completely not done in some parts of the world - some prefer ketchup
In parts of the world it's perfectly legal to carry guns and it's part of their legacy, it's part of their constitution - so I respect that.
What I don't like is my situation being exploited by certain people who are frustrated with the UKOUG.
This certain person - who we will give the fictional name 'Dom' - turns my small incident into a 'vicious attack'… man - I'm lucky to be alive if I read all this. He then goes on into explaining, 'how everybody is against Americans'… he and his wife were verbally threatened when they were once abroad (they were called 'yanks').
Well - what can I say - I'm pretty sure that's not a pleasant situation when you're in a foreign country, I can't deny that. But I actually happen to like Americans as some of my best and dearest friends are yanks ;-) (Which is not an offensive word where I come from - but again cultures differ)
Anyway - 'Dom' has done a pretty good job filling in all the blank spots of my story, took out all the irony out of it, sprayed some assumptions on top of it, and came to the bizarre conclusion that all overseas conferences are very dangerous places to go to - and UKOUG in special.
The fact that I got attacked, has nothing to do with the UKOUG. It could have happened anywhere. But thinking of it, I'm kind of glad it happened in Birmingham. Birmingham, like a lot of UK cities has a vast network of CCTV cameras. So the police knew perfectly what had happened. When I would have chosen to press charges, police could have easily arrested my attacker just by following his whereabouts on the CCTV cams. And they came in between before serious things happened, I was pretty chuffed about that.
'It's a scary feeling being disarmed and knowing that you are defenseless in a foreign land. In many countries you cannot even carry a knife to protect yourself, and Americans attending Oracle conferences abroad are at the total mercy of criminals. But it's not just Americans who get singled-out for attacks.' (Dom)
Duh … what can I say … I'm speechless.
Ok let's assume for a minute I was attacked in the US at some Oracle conference. I probably would have reacted identical, as quite honestly, my adrenaline was pumping harder than a Bee Gees remix of 'staying alive'. Following Dom's logic, the attacker in the states would have probably carried a gun or knife - which probably meant, he would have shot me *and* the police would have probably arrested me too. And therefore you should not attend any Oracle conferences in the US Puh-lease!
Or assume you're attending an Oracle course on a cruise ship and the ship is attacked by pirates - what to do … I might just as well stay home then.
The fact is that 'Dom' probably has a score to settle with the UKOUG for whatever reason and he's taking my incident out of context to make his point - 'UKOUG doesn't like me - UKOUG bad '.
Anyway - I'll be attending the UKOUG next year as it is simply the best Oracle user conference out there.
(and it'll stay that way until someone starts a conference at a ski resort - I'm thinking Solden, Austria )
I've been promoting UKOUG among my colleagues and this year the company had sent 7 people (including one of the directors) - which is more then we send to the OOW in San Francisco. So, this incident is *not* going to stop me or the company from coming over. Hey, especially with the British Pound at it's all time low this was a sweet deal (oh -now you're sorry you didn't switch to the EURO ;-) ).
Instead, we had a good laugh, a bucket of ice at McDonalds for my eye, and a pint to wash down my bruised ego
]]>UKOUG 2008 was a blasthttp://www.ora600.be/ukoug2008-was-a-blast2008-12-12T10:13:23+01:002008-12-15T20:08:23+01:00kurtvmIt flew by so fast I didn't feel like wasting my time blogging about it last week. I don't know how the other guys do it, but I had too much fun with other things.
I could give you a detailed description of all the sessions I followed but that would be rather dull
In my opinion there were a couple trends emerging:
1.The rise of Oracle VM
I think this will be one of the biggest things in the near future. It’s a no brainer. Companies are already in love with VMWare – however, anyone who has half a brain will never run an Oracle production database on it, as the combo is simply not certified. In fact, according to metalink note 249212.1, Oracle has not certified any of its products on a VMWare virtualized environment. Oracle support will only provide support for issues that either are known to occur on the native OS, or can be demonstrated not be as a result of running on VMWare.
]]>It flew by so fast I didn't feel like wasting my time blogging about it last week. I don't know how the other guys do it, but I had too much fun with other things.
I could give you a detailed description of all the sessions I followed but that would be rather dull
In my opinion there were a couple trends emerging:
1.The rise of Oracle VM
I think this will be one of the biggest things in the near future. It’s a no brainer. Companies are already in love with VMWare – however, anyone who has half a brain will never run an Oracle production database on it, as the combo is simply not certified. In fact, according to metalink note 249212.1, Oracle has not certified any of its products on a VMWare virtualized environment. Oracle support will only provide support for issues that either are known to occur on the native OS, or can be demonstrated not be as a result of running on VMWare.
This is where Oracle VM comes in. Oracle VM is basically Xen … however, it seems Oracle tweaked it a bit. I was talking to one of the guys from CERN and they have intensely tested Oracle VM and their benchmark had shown that Oracle VM is up to 20% faster than Xen running Oracle RDBMS. And Xen was already faster than VMWare as VMWare uses a propriety hypervisor and Xen’s hypervisor is based around the linux kernel so to speak.
Now what we’re seeing is that Intel (and also AMD) is coming out with specialized chipsets that enables Xen (or Oracle VM) to make use of hardware assisted virtualization (as opposed to paravirtualisation). Intel with its intel VT-x & AMD with its AMD V-series chipsets. This allows for pagetable mappings being done in hardware as well as direct access to io devices by dma and interrupt remapping assisted by the hardware. I don’t want to throw some numbers around here, but word on the street is that the difference of running the rdbms in OracleVM or native is as low as 5% …. Which is really, really good I think.
Hmmm … they’re not at par with IBM’s LPAR technology yet, but sure enough… this gives us serious opportunities for using virtualization on commodity hardware. Especially because ‘some’ Oracle products are already certified on OracleVM. There’s even a certified combo of RAC-OracleVM (ok – it’s only a 10.2.0.4 32bit on OEL5.1 in paravirtualized mode with limitations… but it’s certified! See ML 464754.1) – and that will never ever happen with VMWare !
With quad core intel processors becoming the norm on the market, you could do things like :
-Two 2way quad core intel boxes (8cores)
-4 cores (oh – OracleVM supports hardpartitioning … which is license friendly !) for running an Oracle RDBMS VM
-1 core for an Oracle IAS Infractructur VM
-2 VM’s with 2 cores each for IAS midtiers
-2 VM’s with 1 core each for linux based load balancer using LVIPS/hartbeat/Ldirector
-Use the ability to migrate a VM for HA !
-And still have room left to scale vertically ...
-And the ability to migrate easily to new hardware when the old one is decommissioned
Oh – when I was talking to one of the CERN guys, he told me they had tested the live migration of a VM running a single instance database and it did actually work … with an interruption of 3seconds. Very cool and exciting stuff.
Anyway … we’ll be decommissioning some hardware next year and we’ll probably be migrating some Oracle IAS midtiers away from IBM LPARs to OracleVM’s on intel.
2.The merge of IAS10g & BEA application server into 11g
Another thing you couldn’t get around was BEA’s technology and how Oracle would all merge it in IAS 11g…. or what would be dropped. Some things will come from IAS10g, some technologies will come from BEA. The J2EE server will be Weblogic’s – which is pretty much a no-brainer…
This has some consequences tho … one of them is, opmn and dcm will disappear (as well performance monitoring tools like for example dmstool). Opmn will be replaced by the node manager. However – if you’re using Forms 11g (or discoverer or reports … anyway … the old stuff everybody is using ;-) ) … IAS11g still comes with opmn just for that !
What’s more – and this is hilarious (well – it would be if we didn’t have so much invested in it) – if you are using Forms or Portal in combination with Oracle SSO, you will need an IAS10g infrastructure as well as the complete IAS11g stack. Hmmm …so far for seamless integration. I don’t know what the deal is about SSO … as far as I can see it’s just a J2EE application that needs access to an LDAP directory (like OID). So I reckon you should be able to deploy it in WLS ? Maybe I’m missing something here… well OK … you need Apache1.3 and mod_osso too … but that’s available on IAS11g…
Oh – the good news is that IAS11gR1 will come with all the wizards and migrate assistants necessary to migrate your good ol’ IAS10g to 11gR1. (you’ll need to be on Forms 10.1.2.x to migrate to 11gR1).
Anyway – throughout all the noise I noted down we’ll be using IAS10g for a long time to come … and I’ll be looking forward to 11gR2.
3.ASM
Quite a lot of talk about Automatic Storage Manager or ASM. CERN is storing 550Tb on it … it’s the de facto choice for running RAC. And a lot of the talks were about ASM internals, performance and rebalancing. One I didn’t want to miss for sure was Julian Dyke’s presentation on ASM internals. I’m a big fan of Julian’s presentations as they are very visual. They have a lot of animations in it and if you’ve ever made extensive use of PowerPoint’s custom animations, you might have some idea of how much time he invests in creating his slides… I’m thinking in ‘weeks’ here!
One interesting fact you will learn from watching his animations is the amount of IO that is involved in rebalancing disks when a disk is added. I believe this was also mentioned in the CERN presentation. Just look at Julian’s work and it will become clear…
One of the sessions gave us an overview of ASM and ZFS. I don’t really see all that many benefits running an Oracle RDBMS on ZFS because (a) it doesn’t support direct IO, (b) it doesn’t support async io and (c) because ASM is more widespread and on more diverse platforms it will shed all possible bugs more quickly. There were other issues too, which were pointed out by Harald Van Breederode regarding extending datafiles on zfs in combination with instance crashes at which case you would end with corruption and at that point Oracle will happily refer you to the ZFS team to fix it. Good luck with that
I almost forgot to mention that I did a presentation myself. Adventures using advanced queueing in our retailing solution… this was based on a presentation I did in Brussels earlier this year. And originally it was a more business oriented presentation. I was quite surprised it got selected for the UKOUG – so I altered it to be more technical and more case oriented. It starts out high level by setting the scene of the retailing business. Then, it goes deeper in the architecture of a case, and finally comes to some of the problems we’ve encountered. I think I should have focused more on the problems and less on the business side. I also should have left ‘retail’ out of the title
Anyway – the people who did show up, asked some good questions. One question was like ‘do your queue’s ever stop queueing’ – and my immediate response was ‘no – to my own surprise – we never have that problem although in the Oracle8.x days we had it a lot’ … afterwards I realized that this was not entirely true as we did have some queueing ‘stops’, if you wish, early on when there was a network failure to one of the remote queue’s … and we do have the occasional orphaned spinning job queue process that we have to kill. But since we’re on release 10.1.0.5 this baby spins like a kitten.
Apart from all the technical content, UKOUG is a good place to meet old and new friends. So I met up with Doug Burns, Daniel Fink, Marco Gralike … oh and I’ve finally met Paul Vallee, who’s one of the coolest dude’s out there. Dan gave me his presentation ‘Never a dul moment’ which he did for RMOUG last month which looked really good. This year there also was an Oaktable setup – however, I only stumbled on it by accident when I was looking for a place to sit down to finish my lunch. I only knew it was the Oaktable as Mogens Norgaard was stuck to it …
I also made some new friends on Birmingham’s Christmas market – kids with hoodies high on drugs. (I reckon they were around 18y of age... or at least old enough to drink!). At one point we went to the market at around 7PM and when we crossed some kids, one of them hit me in the face for no reason. I still had my hands in my pockets so I couldn’t react, and thus his fist came on hard. I guess he was bored or he probably had one pill too many. He also made the mistake that he hit me as I’m 210lbs and I don’t believe in turning the other cheek. And I was bored too – so I immediately ran after him and knocked him down, jumped on him, strategically placing one knee in his ribs and one in his neck. I was almost starting to have fun until 2 of his buddies jumped on me from behind. So my posse and his posse came to a clash and it ended when police in civilian came between us, at which point the ‘other team’ ran away.Some people were gathering around us now – it almost seemed like some elderly people were glad some these kids got disciplined in their own way. The police asked me if we wanted to press charges – which we didn’t because nothing was broken, nor was anything stolen… I did have a swollen eye and I was bleeding a bit, both nothing serious. I reckon that I did more damage to that other guy then he did me, which made me even more certain not to press charges as police might even arrest me ;-) And these kids nowadays – they laugh the police in the face – they do something stupid, get arrested, but as they are minor, they get off with a warning or whatever and are back on the street an hour later … what’s the point of that ? At least now they got what they deserved and I didn’t have to fill in loads of papers. I wasn’t sure I would mention this in the blog as it might reflect badly on the UKOUG, but it might as well happened in any other city in the world. Ironically, Thomas Presslie warned us just an hour before – watch out for kids with hoodies… I should have listened more carefully ;-)
Anyway – It made for a good introduction for my presentation as I had to somehow explain the black eye
UKOUG 2008 was another great event. Sometimes people asked me - which one is better UKOUG or OOW. And I always answer - UKOUG by far.
]]>UKOUG 2008 - day 0http://www.ora600.be/ukoug-2008-day-02008-12-01T02:59:02+01:002008-12-01T02:59:02+01:00kurtvm
It's almost 3'o clock at night - I arrived earlier today in Birmingham.
By sheer luck - both me and Doug Burns arrived at the Jury's Inn at the same time... needless to say ... we ended up in the hotel bar 5min later where we joined the Pythian crew (including Paul Vallee!).
I gave Doug an ORA600 polo shirt 3y ago - and I noticed the print is totally worn off ... funny enough, it actually works as a gimmick ... it encountered severe corruption.
Here's a picture of Doug & Paul
Anyway - It's great to be here again - but as it happens I got called for a recovery emergency and I'll probably be working all night ...
]]>
It's almost 3'o clock at night - I arrived earlier today in Birmingham.
By sheer luck - both me and Doug Burns arrived at the Jury's Inn at the same time... needless to say ... we ended up in the hotel bar 5min later where we joined the Pythian crew (including Paul Vallee!).
I gave Doug an ORA600 polo shirt 3y ago - and I noticed the print is totally worn off ... funny enough, it actually works as a gimmick ... it encountered severe corruption.
Here's a picture of Doug & Paul
Anyway - It's great to be here again - but as it happens I got called for a recovery emergency and I'll probably be working all night ...
]]>Kugendran Naidoo awarded best speaker and best innovative presentation at SAOUGhttp://www.ora600.be/kugendran-naidoo-awarded-best-speaker-and-best-innovative-presentation-at-SAOUG2008-11-20T10:29:30+01:002008-11-20T10:33:34+01:00kurtvm
It seems that real-life (and larger than life) database recovery stories always seem to lure in the crowds.
It's like a car accident - people slow down, have a look and hope it never happens to them (and causing additional traffic jams while doing so).
I mentioned in my previous blogentry that Daniel Fink will be presenting some real-life recovery stories at RMOUG on the 21th november, so I'm pretty sure it will generate some disaster tourism
One of the other ORA600 partners - South African's NRG Consulting - delivered a similar presentation last month at the South African Oracle Usergroup (SAOUG) :
Kugendran is one of the brightest people I know - so I know he's a good presenter.
But Kugendran sure hit a home run that day.
He received the 'Best speaker award' for day 1 *and* the award for "Best Innovative Presentation 2008" :
Two thumbs up !
]]>
It seems that real-life (and larger than life) database recovery stories always seem to lure in the crowds.
It's like a car accident - people slow down, have a look and hope it never happens to them (and causing additional traffic jams while doing so).
I mentioned in my previous blogentry that Daniel Fink will be presenting some real-life recovery stories at RMOUG on the 21th november, so I'm pretty sure it will generate some disaster tourism
One of the other ORA600 partners - South African's NRG Consulting - delivered a similar presentation last month at the South African Oracle Usergroup (SAOUG) :
Kugendran is one of the brightest people I know - so I know he's a good presenter.
But Kugendran sure hit a home run that day.
He received the 'Best speaker award' for day 1 *and* the award for "Best Innovative Presentation 2008" :
Two thumbs up !
The SAOUG also made all presentations available on their website here.
And they've put a massive amount of pictures online.
Man - this sure looks like a class A event and I regret not being their with Kugendran - but who knows, maybe I'll submit a paper next year.
… so the editorial team must have been geeky enough to download it and let all the animations spin their heads around.
Puh-lease!
