Escolar Documentos
Profissional Documentos
Cultura Documentos
Issue 6
SQL® Magazine
MyS
News, articles and feedback for MySQL database administrators and developers
using MySQL on a daily basis to provide some of the best database infrastructure
available.
17 log-bin
Page 2 Fall 2008 MySQL Magazine
STEPS:
1) The business user would like a new application and develops a requirements list for you
2) You start analyzing the system and write down what kind of data you need to keep. You then start
designing an initial idea for a database.
3) You ask what kind of decisions you need to have for the new application and then make decision tree
diagrams. You can then show the diagrams to the business user to see if you are on the same page. Some
examples:
a) What discount should a new customer get?
b) When is it ok to go outside and play a baseball game (according to the weather)?
c) Does the hotel have available rooms?
4) You can convert the decision tree into a
decision table, which naturally fits database tables
where you can store it:
you can search the decision table for the result to the decision you are looking for and save it to your table.
6) There are also ways to create and search decision tables by minimizing the rows using “All” (meaning in all
cases for this condition) instead of True/False. In the event where you forgot a certain situation, or tree
branch, you can configure your application to add a new row to your decision table for that situation and to
message you to fill it in later. This helps with debugging behaviour.
7) It is important to note that there is an element of Test Driven-Development here. The idea of creating tests
first is to help discover what your program should do, run the test to check if it works and then “flesh out”
the code to make the test pass. Here you use the conditions to help you create tests - Null for fail and
True/False for pass. You also know what the application should do because you planned it when you
1) I already talked about the benefits for developers. It helps the coding process by defining what you
should code and it sets up tests in a way to verify your code. Since you keep records of the results for the
conditions, you have records for debugging.
2) Because of the similarity of decision tables and database tables there is an opportunity to be able to alter
them. By doing so you can change the behaviour of the application. In the events of larger changes, such
as adding conditions and results, you have a lot of control for maintenance and testing.
3) Finally, since the database has more information about your data and how it relates to what you need, you
can produce very fast queries and reports. In the example of the hotel-reservation system, you can do a
search for which rooms are available according to “WHERE available = True”.
4) It is also important to note that the logic is not necessarily held in the database if you are really against
that. The code is the logic which does the “How” and the database holds the results which is the “What”.
Conclusion
I see this method as similar to the way I have always developed my applications, but with a slight difference. I
usually grab data from different parts of the database, process the data, keep the results in memory and then
throw them away when I am done. This time I just decide to keep the results in the database for later.
While using this method, I really felt very surprised at the end of the development process and how easy it
was to get to it. I felt a bit like I was “cheating” while I was working on the code, because I already knew all
the answers. I am very happy with how this idea turned out. I am particularly happy with the testing element
and the maintenance element which I know helps cut down on problems in development and keeps the
application going for a long time.
Tracking logs
In MySQL, the most basic logging method for any table, or set of tables, is to enable binary logging. Given
a valid binary log, the state of any table (or combination of them) at any previous instant can be rebuilt
correctly with the mysqlbinlog utility.
But if that's the only audit trail we have, then every time we run an audit report, we have to regenerate at
least part of the database. This is not practical. An audit trail has to be directly available via straightforward
queries. That requires a tracking log table.
The simplest possible tracking log table has:
1) Every column that exists in the tracked table,
2) An action column to record the data-changing action (update, insertion or deletion),
3) A time-of-change column with the desired datetime granularity
4) A who-changed-this column (optional).
What is the desired datetime tracking granularity? Whatever is guaranteed to distinguish all possible
updates. In most systems including MySQL, two identical modification commands to the same table cannot
be executed in the same second, so change_time TIMESTAMP is reasonable.
After creating the tracking table, we write triggers to save dated tracking copies of all row modifications,
and we prohibit deletions in the tracking table.
This approach delivers two large benefits—it is simple, and all tracking logic remains under the hood,
invisible to an application using the tracked table. The downside is redundancy: every change is written twice,
once to the table and once to its tracking twin. Three times if you count the binary log.
Given the tracking log's primary key, the update and deletion triggers will fail if two identical modification
commands are attempted on the same row in the same second. That side-effect is probably desirable. If it is
not, since MySQL out of the box does not make fractions of seconds available at the query level, you will have
to add the Milliseconds() function (http://www.artfulsoftware.com/infotree/mysqltips.php) to your server, or
define transaction times as integers and populate them from a frontend application which can report time in
fractions of seconds.
USE sys;
CREATE TABLE viewparams (
id int(11) PRIMARY KEY AUTO_INCREMENT,
db varchar(64) DEFAULT NULL,
viewname varchar(64) DEFAULT NULL,
paramname varchar(64) DEFAULT NULL,
paramvalue varchar(128) DEFAULT NULL
);
Time Validity continues on next page
Page 10 Fall 2008 MySQL Magazine
USE test;
DELETE FROM sys.viewparams WHERE db='test' AND viewname='vEmpHist';
INSERT INTO sys.viewparams (db, viewname, paramname, paramvalue)
VALUES ('test', 'vEmpHist', 'date', '2008-9-1' );
Or, until MySQL has proper parameterized Views, we can use sprocs like emp_asof() to retrieve any
previous table state. For example, this is what the employees table looked like on 1 September 2008:
CALL emp_asof('2008-9-1');
+----+----------+-----------+--------+---------------------+---------+-----------+
| id | lastname | firstname | gender | dob | marital | ssn |
+----+----------+-----------+--------+---------------------+---------+-----------+
| 1 | Black | Mary | F | 1972-10-31 00:00:00 | M <--- | 135792468 |
| 2 | Higgins | Henry | M | 1955-02-28 00:00:00 | W | 246813579 |
| 3 | Turunen | Raija | F | 1949-05-15 00:00:00 | M | 357902468 |
| 4 | Garner | Sam | M | 1964-08-15 00:00:00 | M | 468013579 |
+----+----------+-----------+--------+---------------------+---------+-----------+
That’s all for this time. Next issue we will wrap things up with a discussion of transaction state tables.
Peter Brawley is President of Artful Software Development and co-author with Arthur Fuller of Get It Done
With MySQL 5&6.
MySQL, MySQL logos and the Sakila dolphin are registered trademarks of Sun
Microsystems, Inc. in the United States, the European Union and other countries.
MySQL Magazine is not affiliated with Sun Microsystems, Inc., and the content is not
endorsed, reviewed, approved nor controlled by Sun Microsystems, Inc..
Page 11 Fall 2008 MySQL Magazine
Read the Kickfire white paper to find out how to slash hardware buildout.
Page 12 Fall 2008 MySQL Magazine
Installation
Installation requires some homework before
starting.
1) The tools require Perl and need to have
two modules pre-installed: perl-DBI and perl-
XML-parser. You can get these packages either
straight from your distribution repositories or
CPAN.
2) You also need to define a backup user http://forge.mysql.com
within MySQL. The permissions that this user
needs are detailed in the manual. You can create
one user to backup all databases or one user for each database or group of databases. You also may have
different users for backup and restore, or use a SUPER user to perform all these tasks. Probably for smaller
environments, using a SUPER user is OK and for more complex ones you might need to define one or more
backup users.
3) ZRM also requires mailx to email the backup reports to the backup administrator. If you use a package
manager like Debian's apt-get or Red Hat's yum to install from the Zmanda packages, chances are it will be
installed automatically.
4) Make sure that the path to the MySQL utilities are in the PATH environment variable since ZRM uses
several of them.
5) If you use snapshots for the raw backups the associated utilities should also be accessible.
ZRM continues on next page
Page 13 Fall 2008 MySQL Magazine
6) If you are going to do incremental backup and points in time recovery you will need to configure the
server to write binlog files setting the MySQL bin-log option. Remember that setting this option requires
restarting the server.
7) For remote backups using SSL the corresponding utilities and libraries have to be installed as well.
The download and package installation is pretty straight forward. Zmanda offers packages in the most
popular formats or you can install from a tarball. The documentation includes a detailed explanation for all of
these alternatives.
Configuring ZRM
The main concept around ZRM is the backup set . If the databases are midsize and bigger and have
regular traffic, it is not recommended to backup all of them at once, in which case you are going to create
groups of databases and / or tables to backup together. Also you will probably schedule different backup
types with different frequencies at different times. The combination of databases, type of backup and
schedule is what constitutes a backup set .
For my examples I am going to include some of the options to backup the database used by the media
player Amarok (http://amarok.kde.org/). I will define three backup sets:
· Monthly logical backup (music-monthly)
· Weekly raw backup (music-weekly)
· Daily incremental backup (music-daily)
Each backup set is identified by a given name. In
the example above I enclosed each in parenthesis. Can’t Get Enough?
The configuration files included in the installation
are commented in great detail, which makes it really
easy to use them as a reference to create your own
configurations. As you plan the settings for each one
of your sets keep in mind the order in which the
configuration parameters are evaluated. The global
configuration file is over-ridden by the backup set
configuration which in turn is over ridden by the
command line options. The backup set is identified
by a unique name, which is the name of the directory
where it's configuration file is stored.
PARAMETERS OVERVIEW
There are numerous parameters to fine tune the
ZRM configuration. Reviewing some of the basic ones If a quarterly dose of MySQL news and technical
gives us an idea of the tools capabilities: articles is not enough, visit Planet MySQL at
http://planetmysql.org to read the MySQL
ZRM continues on next page
employee and community blogs. There are
several new articles daily; bloggers can submit
feeds as well.
http://planetmysql.org
Page 14 Fall 2008 MySQL Magazine
It is also important to specify the different plug-ins. These specify the utilities that perform copies,
compression, encryption, pre and post backup scripts, etc. This gives the tool a lot of flexibility. For
example, it possible to include the my.cnf file along with the data backup.
Reports
ZRM offers a number of predefined reports and it is also possible to create custom ones. In addition to
these, it is possible to create simple text reports and html. These reports can be stored in a web server
ZRM continues on next page
Page 15 Fall 2008 MySQL Magazine
directory, mailed and/or add to an RSS feed. The reports can be as short or detailed as needed.
Two of the most interesting ones are the backup-performance-info and backup-app-performance-info.
The first one will give you size and time information about the backup. It is a great way to plan for storage
and/or time of day most adequate for doing the backups. The second one will give information that will
affect the application performance, the time information includes for how long were the locks kept and how
much time did the flush logs take.
The following is an example of the html report output:
REPORT TYPE : BACKUP-APP-PERFORMANCE-INFO
backup_set backup_date backup_level backup_size backup_time read_locks_time
Tue 07 Oct
amarok 2008 07:39:36 0 13.02 MB 00:00:03 00:00:01
AM PDT
When you run the backup, the log includes very detailed information including the MySQL utilities it is
running and which which parameters, which makes it useful for diagnosis in case you have any kind of
problems. A copy of the log is stored in the backup set configuration directory.
Selective Restore
ZRM includes a binlog interpreter. With this tool it is possible to browse the incremental backup up to a
given SQL statement. This way it is possible to recover not only from the regular situations, but from human
error as well (ie: skip the TRUNCATE or ALTER TABLE command). It is also possible to restore based on
timestamp or binlog position, which allows easy point-in-time recovery and restoring a slave up to a given
point. There are specific reports to help with these tasks.
Conclusion
The main points I like about this tool:
· Using snapshots will only lock the database for a few seconds, making it a good alternative to
a hot backup utility.
· The plug-ins allow for a lot of flexibility to integrate different tools to perform the different
steps of the backup.
· The number of predefined and custom reports use HTML format and create RSS feeds.
· The Selective Restore is definitively a life saver.
Take it for a spin. The default configuration is easy to interpret and tune for small installation and it
is also easy to customize to the needs of complex installations. It is storage engine independent (for
snapshot backup it will work only with transactional engines), which means it will cover the current
and future engines.
Bookworm
This month I went back to a book that has been around for a while. The book is MySQL Stored Procedures
Programming (O'Reilly, 2006). Clocking in at just over 600 pages it is the definitive guide on programming
stored procedures. O'Reilly consistently puts out quality books and this one was no exception. Guy Harrison
and Steven Feuerstein know their subject matter and it shows.
Roughly half of the book is spent showing how to program custom procedures in SQL. It also covers
programming with other languages such as Perl or PHP. In addition, a significant amount of space is devoted
to showing how to optimize stored procedures. If you need to program stored procedure this book should
definitely be on your shelf.
Log-bin
The end of July brought us the announcement that Brian Aker was essentially forking MySQL with the
Drizzle project. The last two months have brought a flurry of activity with many of the heavyweights of the
MySQL community getting involved. When this announcement was initially made some people wondered
where this was going to leave MySQL.
I have good news. It leaves MySQL (the company) and MySQL server in exactly the same place it was
before the announcement. Drizzle isn't going to "replace" MySQL. Drizzle fundamentally has different
objectives than MySQL server. If anything, cross-pollination between the two projects will most likely improve
MySQL server. While it is too early to determine just where Drizzle is going, the future look brights for this
project.
If you are a MySQL DBA or a system administrator who spends significant time working with MySQL I
would recommend that if you haven't already you begin following Drizzle development. As it becomes more
stable you should probably even take it for a spin. MySQL Magazine will continue to follow the development
of Drizzle and publish as much information as possible about it as I consider it to be an important part of the
MySQL "sphere".
Thanks,
Keith
bmurphy@paragon-cs.com