Cloud Service Providers: Attacking Big Data Old-School Style

As more and more information floods into the Internet, organizing and making sense of this Big Data becomes more important and more difficult.

New database methods are emerging to help process unstructured data, but IT developers and database deployers also have to figure out how to deal with the world of legacy technology.

For the last 40 years, relational database programs (usually powered by SQL-based management systems) have been the backbone of supplying businesses with organized rows and columns of data. The problem is that these legacy systems may not be able to work together to give businesses the information they need when they need it. Older programs may also have trouble processing data requests over long distances.

A New Way Of Thinking About Databases

A new way of thinking is needed. Over the past decade the push for “not only SQL” or NoSQL database software has provided a pathway for businesses to connect bits and pieces of data from a variety of sources at very rapid speeds across different geographies.

(See also What's Next For Taming Big Data?)

Some businesses are spreading out the workloads using noSQL databases within cloud computing-based networks. Others are approaching the problem still using traditional SQL relational database software - and that’s perfectly OK.

Previous articles in this series (Taming Big Data) discuss the benefits of a noSQL database tool like VMware’s vFabric GemFire. But SQL database software retains a well-established community of tens of thousands of developers and integrators who may be reluctant to move beyond the SQL they know and love. What’s a company to do?

For SQL diehards, VMware’s vFabric SQLFire SQLFire is a distributed SQL database typically used for online transactions. The software is more modern than most traditional relational database management systems.

What Can SQLFire Do For Me?

SQLFire functions and performs much like GemFire under the hood. SQLFire uses GemFire's data grid engine, which lets both programs capture data and then replicate and partition the information "in-memory” on the server. But instead of having to learn GemFire commands and controls, SQLFire has a user interface and programming framework that will be familiar to developers used to programming in a SQL interface and with SQL tools.

Backups are enabled through virtual copies on other connected servers, although data can be stored long-term on disks as needed.

Unlike other embedded databases, SQLFire allows several servers to store replicated and partitioned tables, persist data to disk, communicate directly with other servers and participate in distributed queries.

For traditional IT developers and database deployers, the SQLFire interface makes it easier to write applications and take advantage of GemFire’s underlying noSQL technology. Developers and integrators who know SQL well will have an easy time adapting SQLFire to new projects.

SQLFire is perfect for classic Web transactions, especially where there is a need for fast speeds and a requirement to dig deep into clusters of data.

Business Case For SQLFire

In addition to making SQL developers feel comfortable, SQLFire can work across multiple networks and geographies. This comes in handy when enterprises need information at the moment it becomes available on multiple continents.

For example, a large regional bank in the Northeastern United States collects large amounts of data that helps it maintain its regional and branch offices. The bank also monitors customer transactions at tellers and various ATMs.

Bank management was interested in measuring the different types of transactions being handled at each of type of station, what types of accounts they were accessing and the various times of day the transactions took place.

Historically, the bank could attach an individual database to each branch, but in today's global environment the company decided it needed to measure all of these data points at the same time for each office. The company tested vFabric SQLFire against its own systems and found the existing server took 20 minutes to complete the queries while the SQLFire server completed its task in less than a minute.

Deploying SQLFire In The Enterprise

In the enterprise sQLFire is generally found on inexpensive computer servers in database clusters. A typical use case would find SQLFire helping eliminate potential data bottlenecks in new mobile and Web environments. Another common deployment option for SQLFire is to integrate it with existing traditional databases or analytics programs.

The software can also be interfaced through an API using programming languages such as Java or Spring. SQLFire is also compatible with Java database (JDBC) or ADO.NET.

As companies look for new ways to make data accessible and provide a consistent view of that information, it's important to have tools that suit the needs of all kinds of developers and IT managers.

VMware’s GemFire and SQLFire software are designed to address just those needs - allowing companies to move beyond concerns over speed and scale and tackle Big Data applications head on.