Oracloid Blog

Oracle Open World 2009 is just few weeks away and I firmed up my presentation schedule now. I will present three “normal” presentations and couple unconference sessions. I’m arriving in San Francisco few days before the conference (7th of October) get to the Oracle ACE Directors briefing so I’ll spend the first few day in Redwood Shores and then off to Moscone Center.

Before I get to the schedule, if you want to catch up with me during OOW — tweet me @alexgorbachev. You are likely to see me in the OTN Lounge or in “The Cave” if you know what I’m talking about.

Here is a quick summary of my presentations:

My first presentation starts on Sunday as part of the Oracle User Groups Forum. It’s one and a half hour long session and as far as I know it’s the only long slot available on Sunday. The penalty is to start at 8:30am so I’ll see you all fresh that morning! :) I have presented on Sunday last year (also a long session with demos) and it was great success — it was standing room only and I know few people could get in (that’s why I’m repeating the session this year as part of unconference — see below). If you want to make sure to have a spot — use Schedule Builder and register for the session. Last time, I’ve done this presentation as the master class at the UKOUG Conference 2008 and it was very successful session. I will most likely pre-record the demos to make sure we fit in the allocated time — there is lots of content.

The next presentation is on Monday and I’m very much looking forward to that one. While the presentation is about building monitoring plug-ins for Grid Control, the example is based on the MySQL plug-in, which I wrote and that Pythian published as free software and Oracle validated the plug-in working closely with us. I had proposed this session for the last year but I guess I’ve made a mistake of adding the MySQL in the title. :) This year I removed MySQL from the title and the presentation passed through the Oracle Mix voting being in the top 10.

The last of my “normal” conference sessions in on Tuesday at 11:30 — just in time before the lunch break. This is a rather unusual area for me as the talk is about E-Business Suite and how to design EBS environment for high availability. While I’m not an Apps DBA, I’ve been digging into that area quite a bit recently and I’ve done this talk here in Sydney during AUSOUG InSync 09 conference.

Finally, I’m doing couple unconference sessions. Many people asked for a repeat of my last year’s presentation “Under the Hood of Oracle Clusterware” which was a huge success so I decided to put that session into unconference schedule. I have reserved a double slot as there is a live demo and it takes some time. In addition, there is always quite a bit of follow up during this presentation so I don’t mind to get side-tracked and dig into live environment that I will have setup for this demo. In addition, we should check out what’s new in 11g Release 2 Clusterware (re-branded as Grid Infrastructure).

The last unconference session is on Wednesday and I decided to make it less of a presentation but more like a roundtable. I hope that my role would be as just a moderator as I would be looking forward to get more participation from the attendees and share methods and tools that various companies adopted to make their teams highly available. I’m also excited to have Paul Vallee by my side as he has lots of insights on this topic. The idea behind this session is that very little is told about organizing a proper round the clock support team. I’ve seen lots of situations when companies deploy expensive HA solutions without thinking about organizing seamless support from their engineers so it’s “normal” for one or two engineers to be “chained to the desk” and on the hook 24×7 no weekends and vacations. I would be more than happy to bring forward and discuss how we handle it at Pythian but I would like to see more contributions from the attendees and not make it look like a Pythian marketing plug. :) It’s not about DBA support only — it’s just that “HA DBA” sounds really cool!

And don’t forget about Bloggers Meetup this year. I’m organizing it with the help of OTN and it should be good fun this year.

One of the customers (actually a prospect) here in Australia asked me about minimal Oracle licensing on a quarter rack database machine. This prompted a thought of using Oracle Standard Edition instead of full blown Enterprise Edition with bunch of options.

Before even going into possibility of using Oracle SE for the database machine, let’s see if we even want to.

Why Oracle Standard Edition?

If the environment is data warehouse then it’s extremely unlikely that Standard Edition will cut it. Lack of many feature make it non-feasible to use for data warehousing — no partitioning licensing, no parallel query, and dozens more.

Oracle Standard Edition might fit OLTP environments depending on the application design and data volumes. Since Database Machine is made to store large amounts of data, we assume that it makes financial sense to run databases that are quite large. Oracle SE lacks some critical features in order to successfully manage VLDB (Very Large Databases). It’s not impossible and depends a lot on the presence of outage windows, how active is the development life-cycle, availability requirements and etc.

Where Standard Edition seems to fit nicely is consolidation of many small databases when each database is relatively small. Number of Pythian customers are running Oracle SE databases that are hundreds of gigabytes in size. it does require a bit more care and overhead in manageability, performance tuning and, of course, limits in many aspects but it fits their bill nicely.

Compute Servers with Oracle SE

Recall that compute servers are dual socket Sun Fire X4170. This means that Oracle SE license could cover up to 2 nodes RAC (included in SE) with 4 sockets in total. For two compute servers in a quarter rack database machine, licensing Oracle SE costs just US$70,000.

In fact, if you don’t need or want to use RAC, you can even get away with Oracle Standard Edition One (SE1) that can be licensed on a server with up to 2 sockets. Oracle SE1 license costs only $11,600 per compute server then.

Licensing Oracle SE and SE1 is a great way for consolidation. The only problem is manageability but there are ways to achieve efficiencies there (I think I will be writing about it soon).

Exadata Storage Server without EE License?

The next issue is whether Exadata Servers with Sun FlashWire can be purchased and used without Exadata software. Technically, I believe nothing prevents us from doing so. Of course, “brainy” part of Exadata (smart data filtering on storage server) won’t be available but that might not be that important in OLTP environments.

InfiniBand Support Limitations

Here come the bad news… If you review Oracle® Database Licensing Information 11g Release 2 (11.2) document and pay attention to the Feature Availability by Edition, you will find that InfiniBand Support is only available in Enterprise Edition whereas both SE and SE1 marked as not licensed for InfiniBand. InfiniBand Support has the same licensing rules in 10g and 11gR1.

Now, it’s a bit unclear what is mean by InfiniBand Support — maybe it’s related only to the RAC Interconnect protocol? Oracle’s InfiniBand interconnect protocol uses Remote Direct Memory Access (RDMA) while communication between the Exadata storage servers and the Oracle Database is implemented with the Zero-loss Zero-copy Datagram Protocol (ZDP) protocol, which itself based on the Reliable Datagram Sockets (RDS) protocol.

Update: 18-Sep-09: RDS can actually run on top of InfiniBand and TCP. RDMA is the layer below RDS to run it over InfiniBand — so that’s what Oracle must be using for interconnect in the database machine. IP over InfiniBand (IPoIB) is another mechanism for RAC interconnect but it’s much slower than RDS. Thanks Matt Zito for your comment!

If you read our blog attentively, you might recall an interview with Kevin Closson where he stated that “this [Exadata] is Enterprise Edition only”. Well, difficult to argue!

Summary

Buying Oracle Database Machine without licenses is probably a dream. However, one person from Oracle Corp hinted that nothing should prevent us from literally building database machine ourselves from the same components — remember, it’s standard Sun hardware inside. Unfortunately, it defeats one of the benefits of the database machine — it’s been well tested and polished with very specific hardware, firmware, OS settings and etc. In my experience, most of the problems in large systems come from incompatibilities between its moving parts that are sometime very difficult to diagnose properly.

Even if customers can get their hands on a database machine separately from the licenses, chances are that Oracle SE license isn’t appropriate because of limitation on InfiniBand Support feature.

Well, looks like my dreams of Oracle Database Machine on a budget are not going to come true!

Now that I, apparently successfully, predicted OLTP Database Machine on Sun hardware, I had to wake up before 6AM in Sydney to tune into Larry’s joined with Sun Microsystems webcast (just to learn that he is late, by the way – 8 minutes so far…). As the follow up post’s comments show, people are interested in the role of SPARC platform in the new OLTP Oracle Database Machine (turns our there is no role for SPARC as of now).

Waiting… Waiting… ah here it comes — yachts, BMW (yeah love it as well) and Larry walks in — he starts by mentioning lo-o-o-ong partnership with Sun and announced –
“Oracle Exadata version 2 – hardware by Sun and software by Oracle.” Funny, I heard exactly the same sitting at the Oracle Open World last year but with HP. He then proceeds — “It is the *First* Database Machine that does OLTP. All the other machines, Teradata, Netezza, etc. are designed just for data warehousing.”

Interesting that Larry’s speech was very harsh on competition and where it comes to data warehousing, it’s Netezza and Teradata, while in hardware it’s IBM. I need to count how many times Larry said “better/cheaper/faster than IBM” during his announcement.

Ellison also seems to freely use Exadata term for the Database Machine itself which is a bit confusing but the more confusion, the more hype and interest so I guess he must have been asked to do it.

Let’s go into the nitty-gritty… ODBM (Oracle Database Machine) v2 is twice as fast as version 1 when it comes to data warehousing. Compare to version 1, ODBM v2 runs OLTP traffic — “something that Netezza and Teradata can’t do at all… but we can do both [i.e. data warehousing] and we do both very well” said Ellison.

So what’s inside?

Database Servers

Sun Fire X4170 with two Xeon Nehalem processors and this is the state of the art. Nehalem processors are the fastest CPU’s for database workload in case you didn’t know. Each server also has 40 Gb connectivity to InfiniBand switch.

Memory is 3 times faster in the new database server and there more the two times memory now — 72GB per database server. For those of you who still use 5 GB for their SGA’s when the server has 128GB — wake up! The prohibitive overhead of larger SGA is long time gone and all modern releases of Oracle handle large SGA really well.

What’s important is that each server has 4 10Gbit Ethernet ports. OLTP traffic now has bigger pipes to connect application tiers which what was one the main problems why ODBM v1 wasn’t suitable for OLTP traffic.

Exadata Storage Servers

Storage Servers are based on Sun Fire X4275. Again top of the shelf Intel Xeon 5500 Nehalem processors.

Capacity increased — SAS disks are 600 GB (+33%) and SATA disks are 2 TB which doubles what ODBM v1 offered. We all know that big disks are bad but that’s where Sun FlashWire technology comes into play with flash cache on the controllers (5+TB for the full ODBM).

This is what Ellison said about Flash cache — “These are not Flash disks. Make no mistake — these are no Flash disks. This is a memory hierarchy made up of DRAM in our database servers and Flash in our storage servers with very sophisticated algorithms, not simple LRU — we know for doing a sequential scan, we don’t blow out the cache.”

Database Machine — all together

8 Compute Servers — note that they are not Database Servers anymore. That means, customers are expected to be able to run their application and web tiers on those compute servers just as well as Oracle databases. Of course, it’s called the server grid. You have in total 64 Nehalem cores and 400 GB of memory.

14 Storage Servers feature 300+TB of disks (using SATA disks) and more than 5TB of flash cache. The flash cache is built into the disk controllers. Each disk controller has 96 GB of flash. Each Exadata Storage server has 4 of those controllers so you do math (we must be at least as good as Larry in math — “1 million IO’s per second (IOPS) from a single cabinet. Obviously, 2 cabinets is 2 million IOPS. 8 Cabinets is… I can do the multiplications in my head — it’ll be 8 million random IOPS.”). So it’s 5+ TB of disk controllers flash cache — Sun’s FlashWire technology.

Ellison mentioned 880 Gb/sec aggregate throughput for InfiniBand. In reality, 8 compute servers can only do 320 Gb/sec which is a lot and if you are going to use this machine for OLTP — you don’t care as all you interested is random IO’s and the machine is sized to deliver one million random IO’s.

It’s not clear whether customers have complete flexibility on configuration of the database machine but based on Larry’s scale-out ideas, it’s possible. He said — “just plug in another compute server or another storage server or just plug in another InfiniBand networking switch”. The latter can let you expand beyond one single database machine and rack together 8, 16, 32 machine and according to Ellison “you can just go on and on and on”. Petabyte era comes indeed, we are in the Petabyte territory with 3 database machine racks.

1 million random IO’s per second — this is what you can achieve if your whole database fits into Flash Cache. The bandwidth of Flash cache is only 2.5 times faster than disks (50GB/sec vs 21 GB/sec) but don’t forget that we need Flash Cache only for random IO’s. Don’t forget that this is random reads performance. Right now, it’s not clear how write IO’s impact the number.

Ellison also mentioned compression and then flash can fit something like 50TB of the raw data after compression (10x compression for data warehouse). It’s not free but the results are impressive and based on the current adoption, customers are very happy with the new compression that Oracle database 11g provides.

11gR2 with Oracle Database Machine is running in-memory-only faster than all new start-ups specialized on in-memory databases. This is an interesting claim — I’m keen to verify it. According to Ellison, “Exadata is faster than the best of the specialized in-memory-only database query processing systems”. Oracle ran TPC-H data warehousing benchmark for 1TB data in memory and… won, according to Ellison.

IBM

Building the similar system on IBM Power6 Model 595 and 8 DS8300 Turbo storage, will cost you US$10 million.

Two racks of database machine cost $2.3 million only which is less than a quarter of what you would pay for IBM hardware. Putting it another way around — you can spend the same dollars and get 4 times faster hardware. Interesting note from Ellison to this IBM set — “we are fault tolerant, they are not”. Hm… I was quite sure IBM hardware is damn reliable but OK, if Larry says so…

One slide said that Oracle Database Machine is 20 times faster than IBM fastest computer. Not sure how that was calculated?

Ellison also claimed that Massively Parallel Architecture is what Oracle has been pursuing over the last 20 years. Interesting that many keep blaming Oracle for focusing on shared-everything while other vendors using shared-nothing with massively parallel processing. Well, if nobody else can do shared-everything — why not to blame this technology and stamp it as non-scalable. Here is where Ellison mentioned IBM DB2 and SQL Server and that they, obviously, can’t do what Oracle database can.

Pricing

• 1 Exadata Server + 1 Compute Server — $110K
• Quarter Rack — $350K
• Half Rack — $650K
• Full Rack — $1.15M

Obviously, that doesn’t include licensing but you can take over you existing Oracle database licenses. Update 16-Sep-09: Beware of Exadata Software licensing – it’s $10K per disk! Official price list has been updated.

How do you migrate

There is no changes to application that need to be done to migrate to Oracle Database Machine — all your existing applications will just work. The migration time-frame can be very aggressive — up to days really.

And if you need help migrating your application to the new Oracle Database Machine — Pythian, of course, can help you in doing so: migration with minimal downtime, optimal layout and capacity planning, help you transfer your licenses, evaluate potential compression benefits, upgrade your database version up to the latest 11g Release 2 (we were part of the beta program as usual) and all the tasks involved including backups and business continuity as well as monitoring.

In line with my prediction from few days ago, Larry Ellison is announcing the new Database Machine — the new version is targeting OLTP workloads and is based on Sun hardware.

Looks like I just got the date wrong. Oh well, now is the announcement, hype and demo is at the Oracle Open World and shipments are to start upon Oracle-Sun acquisition completion.

So what’s new in Exadata that I didn’t mention in the previous blog post? Ah, right — Sun FlashFire technology. It’s no surprise that the new OLTP version of Database Machine is boosting the IOPS (IO’s per second) capacity by introducing the flash drives. Nothing prevented Oracle to place flash disks into the original Exadata, not from technology perspective.

You should also know that Sun has been investing a lot in its Open Storage concept. This is very similar to what Oracle did with Exadata Server except that Sun initiative is a universal storage solution with ZFS in its core while Oracle Exadata is specialized for Oracle databases leveraging Oracle ASM. What does it tell me? Two leading companies have the opportunity to merge storage technologies. I’ll take that!

I might have underestimated Oracle on how quickly it can leverage Solaris SPARC for the database servers in the OLTP Database Machine (Mark, you might be right). SPARC platform has been an excellent fit for high throughput OLTP systems. Again, we already saw Larry’s commitment to develop SPARC technology so within a month, Sun customers’ mood should be rising higher and higher these days.

I will excuse myself for self-quoting from half a year ago:
… it does look like Sun failed to make money on a number of great technologies and products such as Solaris and Java. Why is another story — there were great deal of speculations from people who must know it much better than me. My conclusion is that IBM is a great money making machine while Sun is the technology making machine.

What about Oracle? Well, I think that so far they have excelled in both – money making and technology making. Isn’t that a wonderful combination?

I must say I’m excited to see that Oracle is picking up all the juicy pieces of Sun technologies and targets to make money on them and not sell them off in bits.

I have just released a new version of the MySQL plug-in for Oracle Enterprise Manager — MySQL plug-in 1.1.1. This is a long overdue bug fix release.

There are no new features implemented (we have another branch in development) but just fixed number of fairly annoying bugs that I was finally able to reproduce.

The download link is on the plug-in’s home page where you can also find a data-sheet and installation guide.

Here are the changes in the 1.1.1 release:

• Tested with Oracle Enterprise Manager Grid Control 10g Release 5 (10.2.0.5)
• Fixed the bug with connections not closed properly
• Fixed bug that caused collection to hang and time-out (Net::MySQL bug — not recognizing a final packet in result-set)
• Fixed bug that caused collection processes to spin on CPU (Net::MySQL bug when zero length packet returned from the socket — very weird why it happens)
• Fixed few bugs in Commands and Executions report – graphs produced errors from time to time
• Removed columns Compression and Tc_log_% in Others metric
• Changed metric Opened_tables into ratio per second

I have tested it on Linux and Windows with MySQL 5.0 and 5.1. Please do post here in the comments to confirm that it works on your release and provide the following info:

• Oracle Grid Control Server (OMS) version
• Oracle Agent version
• Operating Sysytem and version
• MySQL version and the details of the build
• Do you monitor MySQL instance running locally (on the same host as Oracle Agent) or remotely

This will help the whole community and confirm that there are no platform/version specific issues. I will take care of summarizing your comments — don’t be afraid to duplicate the info. Thanks!

Any issues please report here as usual.

Update 16-Sep-09: Apparently, all this was true and you can find more details after the announcement that posted here.

OK. It’s not often that I make predictions these days but this was on my mind for a while so here we go. Mind you, I don’t have any confirmed insider information so it’s based on some assumptions, my perspective on Oracle-Sun acquisition and some vibes I can feel in the air.

The rumors are that Oracle Exadata v2 and Oracle Database Machine v2 are going to be announced within few weeks and my take is that it’s going to happen at the Oracle Open World. I don’t think it comes as a surprise to anyone that it will be configured with Oracle Database 11g Release 2.

Moving on to predictions and speculations…

Oracle Database Machine — beyond data-warehousing

Oracle Database Machine will not be targeting ultimately data warehouse environments — it will attack OLTP applications and marketed extensively for consolidated environments. “Old” HP Oracle Database Machine was an ideal consolidation vehicle but there were few design elements incompatible with OLTP and consolidated environments. For example, very thin communication pipe with the external world — no way enough traffic could enter (and leave). Have you paid attention to what Oracle did in 11g Release 2 with its Grid Infrastructure component and how well it now can truly manage the grid including database, storage and application tiers? 11g Release 2 delivered inside Oracle Database Machine v2 — this is the ultimate answer to data-center consolidation… the ultimate data-center consolidation appliance.

Oracle Database Machine — truly Oracle hardware

Did you see the title is missing something? You would if you compare it carefully with the year old entry — “HP” is missing! Yes, I believe that Oracle Database Machine v2 will be delivered on Sun hardware.

The exact figures are not published but Oracle claims that adoption of HP Oracle Database Machine is high. If Oracle makes Database Machine available for generic use and data-centers consolidation (those are very “popular” projects now for our customers) then demand will surge immensely and Oracle will need to meet supply. Of course, Oracle could let HP pocket quite a bit of cash but why not leave it all for themselves?

Oracle is committed to Sun hardware if you didn’t notice.

Why on earth would Oracle continue to use HP hardware for it’s Database Machine v2? This is rather highly improbable and the natural choice is… Sun Oracle Database Machine!

I reckon that Sun Oracle Database Machine (or simply Oracle Database Machine) will be still based on the same x86 architecture and not SPARC. Oracle Exadata software is generic and will run on Oracle Enterprise Linux on any x86 platform (it even runs inside a virtual machine if one wants to play with it). So the only work left is to polish and integrate components flawlessly as it’s been done with HP hardware.

The step to introduce SPARC platform is not huge — Solaris on SPARC is still one of the most stable platforms that Oracle Database runs on. What’s left is to make Solaris SPARC playing nicely with InfiniBand and Oracle’s DDP (Direct Data Placement) protocol. I don’t think Oracle could go as far as having it implemented it on SPARC platform already now but I think that database servers can easily be running on SPARC in the future as well as Linux x86_64 while Exadata Storage Server will most likely stay on Linux.

As a summary, I want to quote what I said back in April:
What Oracle intends to do now is to keep all those new acquired products, continue development while work heavily on integration solutions. Integration products and services is where Oracle will find its new direct revenue stream.

Comments are most welcome as usual.

Oracle RAC SIG is holding elections for its officers and it’s now time for all RAC SIG members to vote. The voting is closed on 30-Sep so make sure you don’t miss the deadline — go to Oracle RAC SIG web-site and click on “Elections” tab (here is the direct link if it works).

If you are not already a member of Oracle RAC SIG and interested in Oracle RAC technology then you should definitely join — it’s free. As a member of RAC SIG you can enjoy regular webinars about Oracle RAC technology as well as access previous archives. Some time ago, I have done a webinar on RAC connection management and there are many good presenters involved. RAC SIG also active at the conferences and organizes round tables, speaker panels and other events. I’ve been involved into one of the past events — RAC Attack! in Chicago — and also presented RAC session at the last year’s Oracle Open World. I’m doing another RAC presentation at the Open World this year during User Groups Forum on Sunday on behalf of RAC SIG.

RAC SIG keeps growing and engaged into more and more activities. I’ve been involved in RAC SIG activities for a while and now decided it’s a good time to step in more actively so I am running for the Vice President of the RAC SIG this year – I believe I can bring great value and make RAC SIG even more successful. You can find more details at the “Election” page of the Oracle RAC SIG web-site so please have a look and if you think I would be a good candidate (I’m sure you do) — cast your votes appropriately! ;-)

Are you an Oracle blogger attending Oracle Open World 2009?

If so, you are invited to attend this Oracle Bloggers Meetup during OOW 2009 — a chance to meet your online buddies face-to-face in relaxed and informal atmosphere.

What: OOW 2009 Bloggers Meetup

When: Tue, 13-Oct, 6:00pm

Where: Jillian’s — 4th Floor, 101 4th Street, San Francisco

It’s a big disappointment that Eddie Awad is not going to be with us at the Oracle Open World this year… But the show must go on and Oracle Bloggers Meetup must happen again this year so I’m picking up the baton from Eddie and will organize the meetup this year with the help of Justin Kestelyn and Lillian Buziak (Oracle ACE Wrangler).

First things first, thanks to OTN for sponsoring our gathering again — just like the last year, we will have drinks served for a while. But there are some differences from the previous years…

This year, the meetup is NOT hosted in a pub but DON’T PANIC. We’ve got it covered. We have booked the special function room and ordered some snacks and, of course, drinks will be served. The meetup is hosted at Jillian’s Billiards Club on the 4th Floor in Natoma Room with a nice view (so I guess smokers won’t be disappointed). This is just couple blocks from the Moscone Center. Address — 101 4th Street, San Francisco. I emphasize again — it’s not in the pub itself but on the 4th floor in one of the function room.

The Oracle Bloggers Meetup is on Tuesday, 13-Oct starting at 6PM. You probably don’t want to be too late and miss the fun (and food). The venue is booked until 8PM so at this point we will most likely move on.

It will be, obviously, easier to talk to each other than if we host it in the pub. We also plan to have a small fun program and we have some ideas. Of course, the content won’t be decided until the last minute so please contact me with your ideas by email {last_name} at pythian.com.

I have created the event in Oracle Community — please make sure to RSVP so that we can esstimate the number of people coming. Don’t worry if you are not sure just yet — RSVP “Maybe” for now and change it later.

The bloggers meetups during the Oracle Open World were started by Mark Rittman, continued by Eddie Awad last year. They have been great success so let’s keep it this way! To give you an idea, here are the photos from the last meetup courtesy of Eddie Awad.

Looking forward to see you all there!