Ballerina Data Integration

Introduction

Datatable

Data Connectors
SQL Connector
Introduction
SQL Connector Definition

Samples
SQL Samples
Sample 1 : Simple Select - using select
Sample 2 : Simple Insert - using update
Sample 3 : Simple Insert - With Generated keys
Sample 4 : Execute Stored Procedure - using call
Sample 5 : CRUD Service
1. Introduction
Ballerina Data service connectors can be used to connect to various data sources and
expose/manage data. In the current implementation following data sources are supported.
Any SQL RDBMS - via SQLConnector (any RDBMS with JDBC driver support)
Cassandra Connector
MongoDB Connector
2. Datatable
Datatable is a built-in type in Ballerina. Datatable represent a single dataset. Internally,
datatable hold a pointer to a particular result location in the configured database. The user can
iterate through dataset using next method that locate in ballerina.lang.datatables package.
Each iteration will be point to current row in the result dataset.

Datatable will be return as an output response for either select or call action. In both situation,
initial pointer will be point to just before the first record of the result dataset. With the first
execution of the next, the pointer will advance to the first result record and return a boolean as
a true for the method result. After this pointer advance, the user can access to columns of this
record using other available methods like getString, getInt etc. by providing either column index
number or the column name along with datatable instance. This column access with be
restricted to the columns only in current row. If user execute next again, pointer will be
advanced to the the next immediate result record. After it reach to the last record of this result
dataset, next method will return false. So using while loop, the user can iterate dataset
sequentially. Eventhough, result dataset contains the many data rows only the pointed result
row will be loaded to the memory and in a each subsequence call, data will be fetch from the
database one row at a time.

Available native functions that expose by ballerina.lang.datatable package.

next(datatable dt) (boolean)

close(datatable dt)
toJson(datatable dt) (json)
toXml(datatable dt, string rootWrapper, string rowWrapper) (xml)
getBoolean(datatable dt, int columnIndex / string columnName) (boolean)
getInt(datatable dt, int columnIndex/ string columnName) (int)
getLong(datatable dt, int columnIndex / string columnName) (long)
getLong(datatable dt, int columnIndex / string columnName, string type) (long)
getFloat(datatable dt, int columnIndex/ string columnName) (float
getDouble(datatable dt, int columnIndex / string columnName) (double)
getString(datatable dt, int columnIndex/ string columnName) (string)
getString(datatable dt, int columnIndex/ string columnName, string type) (string)
getValueAsString(datatable dt, int columnIndex/ string columnName) (string)
getArray(datatable dt, int columnIndex/string columnName) (map)
3. Data Connectors
3.1. SQL Connector

Introduction
Ballerina SQL Connector allows to connect to SQL based RDBMS and perform following
operations on the database.
Select Data
Update Data - This includes insert/delete/update of data and Update schema using DDL.
Call stored procedures
Batch insert of data

When initializing the SQL connector, all the required DB connection parameters and connection
pooling parameters should be provided as a map of key value pairs. SQL connector can be
initialized using two approaches.

1. Providing the Data source class Name

In this approach the name of the DataSource class provided by the JDBC driver should
be provided with the key of dataSourceClassName. All the datasource specific
properties needs to add with the prefix of datasource. To find the supported datasource
specific properties refer the documentation provided by the given datasource
implementation.

Ex:

map propertiesMap = {"dataSourceClassName" :"org.hsqldb.jdbc.JDBCDataSource",

"dataSource.user":"SA", "dataSource.password":"", "dataSource.loginTimeout":0,
"dataSource.url":"jdbc:hsqldb:file:./target/tempdb/TEST_SQL_CONNECTOR"};
sql:ClientConnector testDB = create sql:ClientConnector(propertiesMap);

2. Providing the JDBC URL

This property directs to use DriverManager-based configuration.When using this

property with "old" drivers, it may need to set the driverClassName property. Note that if
this property is used, still it is possible to use DataSource properties to configure the
driver and is in fact recommended over driver parameters specified in the URL itself.
Those datasource properties need to adde with the prefix of datasource. as earlier.

Ex:
map propertiesMap = { "jdbcUrl" : "jdbc:hsqldb:file:./target/tempdb/TEST_SQL_CONNECTOR",
"username":"SA", " password":"", "maximumPoolSize":1};
sql:ClientConnector testDB = create sql:ClientConnector(propertiesMap);
When initializing the SQL connector connection pool configuration parameters also can be
provided. Following are the supported connection pool parameters. HikariCP is used as the
underline JDBC connection pool implementation and refer the HikariCP documentation for more
details on the properties. [1]

Property Name Default Value

dataSourceClassName none

jdbcUrl none

username none

password none

autoCommit true

connectionTimeout 30000 ms

idleTimeout 600000 ms

maxLifetime 1800000 ms

connectionTestQuery none

minimumIdle same as maximumPoolSize

maximumPoolSize 10

poolName auto-generated

initializationFailTimeout 1

isolateInternalQueries false

allowPoolSuspension false

readOnly false

registerMbeans false

catalog Default: driver default

connectionInitSql none

driverClassName none

transactionIsolation driver default

validationTimeout 5000ms

leakDetectionThreshold 0

[1] https://github.com/brettwooldridge/HikariCP
SQL Connector Definition

package ballerina.data.sql;

public struct Parameter { //To define parameters for sql stmts and stored proc
string sqlType; //SQL Type of input data Int,String,Blob,Timestamp,DateTime etc.
any value; //Value of the input parameter can be int, string etc
int direction; //0 -In, 1- Out, 2 - In/out
}

connector Connector (map options) {

//For SQL Select operation
native action select(Connector c, string query,
Parameter[] parameters) (dataframe resultSet) throws exception;

//For SQL Insert,Update,Delete operation

native action update(Connector c, string query,
Parameter[] parameters) (int updatedCount) throws exception;

// For SQL Insert,Update,Delete operation which returns generated keys

native action updateWithGeneratedKeys(Connector c, string query,
Parameter[] parameters) (int updatedCount, String[] key) throws exception;

//Similar to above - with generated key column names can be provided

native action updateWithGeneratedKeys(Connector c, string query,
Parameter[] parameters, string[] keyColumn) (int updatedCount, String[] key)
throws exception;

//For Stored Procedures

native action call(Connector c, string query,
Parameter[] parameters) (dataframe resultSet) throws exception;
}
4. Samples
4.1. SQL Samples

Sample 1 : Simple Select - using select

import ballerina.data.sql;
import ballerina.lang.datatables;
import ballerina.l ang.system;

function main (string[] args) {

string name;
int id;

map connProperties = {};

connProperties["jdbcUrl"] = "jdbc:mysql://localhost:3306/Employees";
connProperties["username"] = "root";
connProperties["password"] = "root";
sql:ClientConnector dbConnector = create sql:ClientConnector(connProperties);

sql:Parameter param_age = {sqlType:"integer", value:"50", direction:0};

sql:Parameter param_city = {sqlType:"varchar", value:"NY", direction:0};
sql:Parameter[] params = [param_age, param_city];
string query = "SELECT FirstName, PersonID FROM Customer WHERE Age = ? and City = ?";

datatable dt = sql:ClientConnector.select(dbConnector, query, params);

while(datatables:next(dt)) {
name = datatables:getString(dt, "FirstName");
id = datatables:getInt(dt, "PersonID");
system:println("Id: " + id + " Name: " + name);
}
datatables:close(dt);
}
Sample 2 : Simple Insert - using update
import ballerina.data.sql;
import ballerina.lang.system;

function main (string[] args) {

map connProperties = {};
connProperties["jdbcUrl"] = "jdbc:mysql://localhost:3306/Employees";
connProperties["username"] = "root";
connProperties["password"] = "root";
sql:ClientConnector dbConnector = create sql:ClientConnector(connProperties);

sql:Parameter param_age = {sqlType:"integer", value:"40", direction:0};

sql:Parameter param_firstname = {sqlType:"varchar", value:"Peter", direction:0};
sql:Parameter param_credit = {sqlType:"double", value:"5000.5", direction:0};
sql:Parameter param_city = {sqlType:"varchar", value:"CA", direction:0};
sql:Parameter[] params = [param_age, param_firstname, param_credit, param_city];
string query = "Insert into Customer(Age,FirstName,CreditLimit,City) values (?,?,?,?)";

int updatedCount = sql:ClientConnector.update(dbConnector, query, params);

string status = updatedCount + " record(s) inserted.\n";
system:println(status);
}
Sample 3 : Simple Insert - With Generated keys
import ballerina.data.sql;
import ballerina.lang.system;

function main (string[] args) {

string generatedKey;
int updatedCount;

map connProperties = {};

connProperties["jdbcUrl"] = "jdbc:mysql://localhost:3306/Employees";
connProperties["username"] = "root";
connProperties["password"] = "root";
sql:ClientConnector dbConn = create sql:ClientConnector(connProperties);

sql:Parameter param_age = {sqlType:"integer", value:"40", direction:0};

sql:Parameter param_firstname = {sqlType:"varchar", value:"Peter", direction:0};
sql:Parameter param_credit = {sqlType:"double", value:"5000.5", direction:0};
sql:Parameter param_city = {sqlType:"varchar", value:"CA", direction:0};
sql:Parameter[] params = [param_age, param_firstname, param_credit, param_city];
string query = "Insert into Customer(Age,FirstName,CreditLimit,City)
values (?,?,?,?)";

updatedCount,generatedKey = sql:ClientConnector.updateWithGeneratedKeys(dbConn,
query, params);
string status = updatedCount + " record(s) inserted.Generated key: "+
generatedKey + "\n";
system:println(status);
}
Sample 4 : Execute Stored Procedure - using call
import ballerina.data.sql;

function main (string[] args) {

map connProperties = {};
connProperties["jdbcUrl"] = "jdbc:mysql://localhost:3306/Employees";
connProperties["username"] = "root";
connProperties["password"] = "root";
sql:ClientConnector dbConn = create sql:ClientConnector(connProperties);

sql:Parameter param_age = {sqlType:"integer", value:"40", direction:0};

sql:Parameter param_city = {sqlType:"varchar", value:"CA", direction:0};
sql:Parameter[] params = [param_age, param_city];

Datatable dt = sql:ClientConnector.call(dbConn, "{call findPerson (?, ?)}", params);

while(datatables:next(dt)) {
name = datatables:getString(dt, "FirstName");
id = datatables:getInt(dt, "PersonID");
system:println("Id: " + id + " Name: " + name);
}
datatables:close(dt);
}
Sample 5 : CRUD Service
import ballerina.lang.datatables;
import ballerina.data.sql;
import ballerina.lang.messages;
import ballerina.lang.jsons;

@http:BasePath("/employees")
service Employee {
map connProperties = {};
connProperties["jdbcUrl"] = "jdbc:mysql://localhost:3306/Employees";
connProperties["username"] = "root";
connProperties["password"] = "root";
sql:ClientConnector empDBConnector = create sql:ClientConnector(connProperties);

@http:GET
@http:Path("/")
resource getAllEmployees(message m) {
sql:Parameter[] params = [];
datatable dt = sql:ClientConnector.select(empDBConnector, "select * from
employees limit 1000", params);
json payload = datatables:toJson(dt);
message response = {};
messages:setJsonPayload(response, payload);
reply response;
}

@http:GET
@http:Path("/{employeeID}")
resource getEmployee(message m, @http:PathParam("employeeID")string employeeID) {
sql:Parameter para1 = {s
qlType:"integer", value:employeeID, direction:0};

string query = "select * from employees where emp_no=?";

sql:Parameter[] params = [para1];
datatable dt = sql:ClientConnector.select(empDBConnector, query, params);
json payload = datatables:toJson(dt);
message response = {};
messages:setJsonPayload(response, payload);
reply response;
}

@http:POST
@http:Path("/")
resource insertEmployee(message m) {
json requestPayload = messages:getJsonPayload(m);

string birthdate = jsons:getString(requestPayload, "$.birth_date");

string firstName = jsons:getString(requestPayload, "$.first_name");
string lastName = jsons:getString(requestPayload, "$.last_name");
string gender = jsons:getString(requestPayload, "$.gender");
 string hiredate = jsons:getString(requestPayload, "$.hire_date");

sql:Parameter para2 = {sqlType:"varchar", value:birthdate, direction:0};

sql:Parameter para3 = {sqlType:"varchar", value:firstName, direction:0};
sql:Parameter para4 = {sqlType:"varchar", value:lastName, direction:0};
sql:Parameter para5 = {sqlType:"varchar", value:gender, direction:0};
sql:Parameter para6 = {sqlType:"varchar", value:hiredate, direction:0};
sql:Parameter[] params = [para2, para3, para4, para5, para6];
string query = "Insert into employees(birth_date, first_name, last_name,
gender, hire_date) values (?,?,?,?,?)";
sql:ClientConnector.update(empDBConnector, query, params);
message response = {};
string payload = "Data Insertion Successfull ";
messages:setStringPayload(response, payload);
reply response;
}

@http:PUT
@http:Path("/{employeeID}")
resource updateEmployee(message m,
@http:PathParam("employeeID")string employeeID) {
json requestPayload = messages:getJsonPayload(m);
string firstName = jsons:getString(requestPayload, "$.first_name");
string lastName = jsons:getString(requestPayload, "$.last_name");
sql:Parameter para1 = {sqlType:"varchar", value:firstName, direction:0};
sql:Parameter para2 = {sqlType:"varchar", value:lastName, direction:0};
sql:Parameter para3 = {sqlType:"integer", value:employeeID, direction:0};
sql:Parameter[] params = [para1, para2, para3];
string query = "update employees set first_name = ?, last_name = ? where
emp_no=?";
int updateCount = sql:ClientConnector.update(empDBConnector, query, params);
string countStr = updateCount + " record(s) updated.\n";
message response = {};
messages:setStringPayload(response, countStr);
reply response;
}

@http:DELETE
@http:Path("/{employeeID}")
resource deleteEmployee(message m,
@http:PathParam("employeeID")string employeeID) {
sql:Parameter para1 = {sqlType:"integer", value:employeeID, direction:0};
sql:Parameter[] params = [para1];
string query = "delete from employees where emp_no=?";
int updateCount = sql:ClientConnector.update(empDBConnector, query, params);
string countStr = updateCount + " record(s) deleted.\n";
message response = {};
messages:setStringPayload(response, countStr);
reply response;
}
}