Analytical Support for Decision Making Assignment

Group Hashtag
201266929 201273815 201269197 201285376 201273669

Contents
Introduction ............................................................................................................................................................ 3 The Proposition .................................................................................................................................................. 3 The Problem ....................................................................................................................................................... 3 The Proposal ....................................................................................................................................................... 3 PART A DATA ANALYSIS AND RECOMMENDATIONS ............................................................................................ 4 Executive Summary................................................................................................................................................ 4 Data Cleansing & Preparation ................................................................................................................................ 6 Background ........................................................................................................................................................ 6 Reorganising the data set ................................................................................................................................... 6 Initial work on the data set ................................................................................................................................ 6 Data Selection (including variable selection) ..................................................................................................... 7 Generating additional data ................................................................................................................................ 8 Visual Presentation ............................................................................................................................................ 8 Analysis of Data ...................................................................................................................................................... 9 Exploratory Analysis using Summary Statistics .................................................................................................. 9 Advanced Analysis Inference & Regression ................................................................................................... 13 Recommendations for Head of Department and Minister of Education ........................................................... 15 Part B .................................................................................................................................................................... 17 Value Tree ........................................................................................................................................................ 17 Scoring .............................................................................................................................................................. 18 Analysis ................................................................................................................................................................. 18 Structuring the Problem ................................................................................................................................... 18 Criteria ......................................................................................................................................................... 18 Alternatives .................................................................................................................................................. 19 Uncertainties................................................................................................................................................ 19 Stakeholders ................................................................................................................................................ 20 External / Environmental ............................................................................................................................. 20 Weighting ......................................................................................................................................................... 21 Tier 2 ............................................................................................................................................................ 21 Tier 1 ............................................................................................................................................................ 22 Outcomes ......................................................................................................................................................... 23 Weight Sensitivity Analyses .............................................................................................................................. 25 Recommendations ....................................................................................................................................... 26 References ............................................................................................................................................................ 28

Appendix.29

Introduction
The Proposition
We are a group of students working for the Department of Education in Indonesia and our Line Manager, the Head of the Department, has asked us conduct a quantitative analysis of Indonesia with regards to its educational performance. Once we have analysed the data we are to provide a detailed report for our Manager, who will then use this report to brief the Minister of Education. This report will provide necessary information to the Minister in order for him to make informed decisions on the future of education in Indonesia.

The Problem
Whilst our department provides guidance to the Minister of Education on social and economic issues, that may affect the success and delivery of Education in the country, we have been asked to conduct our analysis over several broad areas, utilising the data from the World Bank. These include: Comparison of Indonesia with surrounding countries in the region Comparison of Indonesia with developed countries Comparison through time

The Minister has also asked us to include exploratory and inferential analysis within our report in order for him and his department to better understand the process of data management so that he may use this model as the basis for all future analysis going forward. The biggest challenge that we face is trying to use and analyse the data provided by the World Bank. Much of the data is either incomplete or irrelevant. Through a process of data cleansing we will provide meaningful and coherent analysis for the Minister in his decision making process for education in Indonesia.

The Proposal
The report is split into two main sections to make the findings of the report more easily understood by the Minister and our Head of Department. The first section of the report looks at the process of data cleansing, explaining how we converted the data from the World Bank into a more meaningful, manageable database and how we established the variables to be considered and our preparation process for using the data. We then discuss how we selected, used and interpreted the visual displays of data and our use of summary statistics in our analysis. Finally we summarise our selection and interpretation of the data using inference, regression, normal distribution and confidence intervals. 3

We finalise this section of the report by providing some detailed recommendations to the Minister in order to help him draw some meaningful conclusions regarding education in Indonesia. The second part of the report discusses the Ministers options for the future of the country and his desire to increase the number of graduates with analytical skills in Indonesia. He firmly believes that Indonesia should have postgraduates with strong core subjects and has identified three key options that are open to him: Option A Send students to study abroad Option B Bring teaching into Indonesia Option C Develop courses within Indonesia.

The chosen outcome of these options will clearly have major political, social and economic implications and as such, the decision must be the correct one for the Minister and his department. For this part of the report we used a Multi Criteria Decision Analysis software tool called VISA. Through a series of stakeholder role play scenarios, lengthy discussions and several variations of scenario mapping, we have provided suitable information to the Minister that should help him in his decision for the future of postgraduates in the country. By using this software tool we were able to demonstrate the potential risks, successes and wrong decisions in all scenarios and therefore the Minister would be suitably informed of all the variable choices open to him and his department. We complete this section of the report with our recommendations to the Minister based on the scenarios and variables that we have demonstrated to him and his department. All supporting data and evidence is attached in the Appendices and has been referenced throughout the report.

PART A DATA ANALYSIS AND RECOMMENDATIONS Executive Summary

The report provides the details of the analysis conducted for the Ministry of Education to provide an insight into factors associated with the enhancement of education. The analysis was conducted by comparing the selected indicators of Indonesia against an equal number of regional and developed countries. In order to undertake a comprehensive review, data for countries and indicators was utilised from the World Bank (The World Bank Group, 2013). Indonesia was compared with the selected countries on identified indicators using the most available current data. The techniques utilised for the analysis were: Visual Analysis Bar Charts Visual Analysis Trend Charts Ranking Tables using a balanced scorecard approach Normal Distribution Regression 4

Visual analysis of bar charts was used to analyse the position of Indonesia compared to the selected countries, while trend charts showed the data pattern of the indicator. The indicators used for the analysis could not completely facilitate the use of ranking tables. Some of the indicators provided were disproportionate for comparison. However valuable information was gained on attempting to analyse the ranking table. The use of normal distribution allowed certain assumptions to be made regarding the current standing of Indonesia with respect to the overall population for a key indicator. Linear regression analysis was performed to determine the level of relationship between two educational indicators. Indonesia was ranked top among the regional countries with respect to primary completion rate and had caught up with the developed countries. We would suggest this is due to the government policy of compulsory education for the first nine years. The focus on spending on education is however in substantial decline. Spending policy does not follow a pattern in regional countries while the developed countries have a pattern. Highly volatile trends for some indicators appear to demonstrate a lack of long term strategy from the Indonesian government. Indonesia's public spending on education as % of GDP is in the bottom quartile of the world population requiring the Ministry of Education to review this figure and make strategies to improve this. Although Indonesia has maintained a pupil teacher ratio competitive to developed countries there is a requirement to introduce advanced training and teaching programmes into education. The number of students attending secondary education is in decline. Qualitative research will provide insights into the reasons behind the decline in the number of students attending secondary education despite the fact this being mandatory for the first three years (AngloINFO, 2013). It is critical for the analysis to be conducted on current, consistent and transparent data on indicators directly related and supporting education. The report is limited to quantitative analysis. Qualitative analysis on the indicators could provide an insight into the reasons for the trends observed and complement the recommendation. It is worth noting that data for certain indicators could be disputed. For instance, some percentage figures were over 100%, even though based on the indicator description, this would be mathematically impossible. As a result, it was important to constantly question the data and conduct the analysis with a level of scepticism. Therefore, it is advised that the recommendations made are also treated with care.

Data Cleansing & Preparation

Background
Data cleansing is a process of reviewing data in its raw form and identifying, correcting and removing corrupt or inaccurate data. Inconsistencies and anomalies may be the result of incorrect data being reported, data not being available for certain countries or may be due to corruption in the transfer of data into the database. A significant amount of data was provided and this had to first be configured into a manageable and meaningful format, before it could be accurately grouped and analysed. Therefore, the first objective of the process was to reconfigure and present the data in a consistent and user friendly manner to allow analysis. The raw data was supplied for a number of countries in the East Asia region for a variety of different indicators.

Reorganising the data set

The initial data set was in a format that was difficult to manage. To overcome this, data was sorted into filter fields to create the ability to select specific data samples which could be easily compared and measured against. The data was then sorted, which enabled the grouping of key indicators into alphabetical order. This brought together the data from each country, for each key indicator, into one block, making it easier to compare. This was further enhanced by adding an empty row between each set of indicators, which helped to visually highlight each section. Finally, the number format of the data was changed to two decimal places to make the figures more manageable. The results of this process can be seen in the Excel Appendix on the Orange worksheet (Hashtag, 2013).

Initial work on the data set

When we first looked at the data set, we recognised that: A number of the key indicators did not seem relevant to the task A number of countries had no data against a large number of the indicators, especially between 1991 and 2000 The only data provided for a developed country outside of the region was for the UK

By referring to the World Development Bank (WDB) data in Google Visualization we were able to identify additional indicators that were relevant to education. Furthermore, by using the WDB map and through trial and error of various analogies between Indonesia and several countries against different educational indicators, we identified a number of developed countries across the world, to provide a comparison point for Indonesia. This gave us 14 countries (regional and developed) and when added to our initial dataset, 37 possible indicators to work with. The new data was put into the same format as the original dataset and the years 1961-1990 removed to make both sets of data comparable. In the original data set, any seemingly unrelated indicators were removed. Finally, both sets of cleansed data were combined in a new spread sheet and separated into sections (Developed and Region). Indonesia was copied into the top cell of each section to allow for easy comparison in visual presentation. This cleansed data can be seen in the Excel Appendix on the Red worksheets.

Data Selection (including variable selection)

It was recognised that 37 indicators was too many to be used to provide manageable amounts of data for analysis and so this needed to be reduced. This was done by relevance of the indicator to this specific task and by the availability of data, across a number of countries. Four categories were established to help narrow down the indicators. These were: Spending on education Areas of spend e.g. teachers Measurable performance outcomes Alternatives to public education

Through this process, the number of indicators was reduced to 17. The next step involved cross checking the available data for these indicators across each country. There was a very low availability of consistent data for 1991-2000 and as it took time for the data to be gathered and reported, there was also minimal data from 2011-2012. The majority of the data set was most complete from 2000-2010 and so for the sake of comparison; we agreed to focus on this time frame. However, there were still indicators that had insufficient data attached to them. For example, data for levels of adult literacy would have been very useful for analysis but the data was sporadic at best across all countries and not reported at all for Indonesia. For this reason, we were unable to utilise it as a key indicator. By applying the same rationale to all indicators, we were able to further reduce the total number of key indicators to 13. Figures 1 & 2 below summarise the final regional & developed countries chosen as well as the list of indicators which we felt were most relevant and had the best quality of data:
Regional Comparison Countries China Hong Kong, SAR Japan Malaysia Philippines Singapore Developed Comparison Countries Finland Germany New Zealand United Kingdom United States Indicator Expenditure per student primary education % per capita GDP Expenditure per student secondary education % per capita GDP Expenditure per student tertiary education % per capita GDP Public Spending on Education % GDP Public Spending on Education % Government Expenditure Primary Completion rate (%) Primary Teacher numbers Secondary Education Teacher numbers Secondary Education Student numbers Pupil Teacher Ratio primary education Pupil Teacher Ratio secondary education Progression to Secondary Education Secondary Education Vocational Student numbers Figure 2 Chosen Educational Indicators

Figure 1 Chosen comparison countries against Indonesia

Whilst we wanted to compare the most current data, the gaps in the data set made this difficult. To overcome this we decided to select the most up to date, most complete data for each key indicator, from the data set. For example, when considering the indicator Primary completion rate, total (% of relevant age group) the year 2010 was selected as this column provided the most data to work with and therefore would create the most meaningful comparison for analysis. Each key indicator was given a separate tab in the excel file and the data for all countries (categorised into regional and developed) was copied into each sheet. This created a workbook with the data organised and categorised into relevant, easy to read sections.

Generating additional data

Using this newly created data set, a number of additional calculations were completed to support the analysis. This included finding values for the: Mean and median for each country (per key indicator). We ultimately decided that the mean would be the most useful and calculated this to give us an indication of the average. Standard deviation (to ascertain the variance of the data from the mean). Ranking of countries. This was a useful way of ordering each country in terms of current position (by key indicator) to compare against other countries. This was done for each group of developing and regional countries and for the sample as a whole.

This process involved a level of trial and error; with some calculations not providing meaningful results depending on the quality of data in the corresponding data set and where size of a country meant it was not easily comparable with others. The creation of graphs and charts was essential for analysing the data and helped with the identification of trends and issues.

Visual Presentation
The visual presentation of data served two purposes; firstly, to aid the analysis and secondly, to allow clear communication of the findings, highlighting key points and aiding understanding of those being informed. By generating visuals of the data, it was easier to check for accuracy. For example, when viewing a series of graphs relating to expenditure per student, it became evident that one country was significantly greater than any other. This was cross checked with the data set and although the values matched, it was recognized that inaccurate reporting may have occurred. In terms of the analysis, using the interactive charts from Google Visualization was very beneficial for quickly bringing data to life, especially over a large time period. This helped us to select which areas to focus on for analysis. Through the creation of graphs with two Y-axes, it was possible to visibly compare, understand and analyse data. Utilizing tools on Excel, it was also possible to predict trends and possible convergence points in the future. Due to the number of countries and years initially identified, the majority of the graphs generated presented too much data and so were incoherent and difficult to compare and to analyse. To overcome this, we reduced the number of countries being compared and adjusted the timeline to increase the amount of consistent data available per country. Through the analysis of the data set, a significant number of graphs, charts and tables were generated, both as snap shots of the position in a particular year and as a trend over time. This was very helpful for analysis, but it was recognized that to include this level of information in a report would be confusing for the reader and that it may dilute the message being conveyed. Whilst all data was relevant, the most important data (positive and negative) was added to the main body of the report, with supporting information added to the Excel appendix. There were a number of other charts that were discarded altogether, despite the time invested in generating them. The lack of available data in some areas had a negative effect on some of the charts, causing confusion when looking at them. For aesthetic reasons, it was decided to remove the corresponding countries from the charts/ graphs however; these were retained in the data sets as evidence that they had not been

deliberately omitted to create a false picture. As a final aid to those reading the report, Indonesia was represented by the colour black in all charts/ graphs, to avoid confusion.

Analysis of Data
Once the data had been properly cleansed to the countries and indicators that we were interested in, it was now appropriate to analyse the data in two ways: Firstly, to compare and contrast the indicators at both a regional level (i.e. looking at how Indonesia compared with other countries in the surrounding region) and on a global scale (specifically how Indonesia compared with developed countries around the world). This would involve the identification of trends between these countries and how they varied over time. It would also involve the comparison of countries in a particular year to see how they performed. Secondly to use a variety of statistical tools. This consisted of inferential analysis to see if assumptions could be made regarding Indonesias performance in relation to the total population and specifically, trying to understand if any of the data could be applied to a normal distribution to make assumptions regarding its performance relative to the overall population. Finally, regression analysis was undertaken to understand if there were linear relationships present between any two indicators for Indonesia and therefore if we could make assumptions on what would drive improvements in any particular indicators.

Successful analysis of both of the above would enable us to make recommendations to the Head of Department and the Minister of Education for recommended improvements.

Exploratory Analysis using Summary Statistics

The first part of the analytical process involved exploratory analysis of the data to present key findings in relation to Indonesias performance in the key educational indicators versus regional and developed countries. Comparison of Regional & Developed Countries

The best way to compare Indonesias performance with other countries over time was using line charts to view trends. In order to compare performance across countries at a particular point in time, bar charts were created. These were ranked from high to low so that Indonesias rank within the sample could be clearly identified. The detailed results of this analysis for each of these indicators can be found in the Excel Workbook with each of the relevant tabs highlighted in Green. For the purpose of this report, we selected 6 key indicators which would give the Minister of Education a balanced view of Indonesias comparative performance by choosing 2 indicators where Indonesia appeared to perform better 9

than its peers, 3 indicators which indicated areas of concern and 1 indicator which on the surface showed signs of positivity with an underlying concerning trend. These are summarised below. Indicators showing positive trends for Indonesia versus peer countries Indicator 1 Primary completion rate The bar chart below shows Indonesia in a positive light for the indicator Primary Completion Rate. Based on the 2009 World Bank data, it has the highest ranked value among both the surrounding regional countries and the developed countries. This can be seen in Figure 3 below:

Figure 3 Primary Completion Rate comparison

The trend charts in the Excel Workbook also show quite a sharp upward trend relative to these countries and from this we can assume that there has been a strong focus to improve results in the primary education sector in Indonesia over the past 6 years. This is an important fact to consider in that it also determines the long term demand for a good tertiary education system and employment opportunities for this population in the long term. Indicator 2 Pupil teacher ratio When looking at this indicator, it was recognised that the large majority of countries, including Indonesia, showed a downward trend over time. In the context of this indicator, this was viewed as a positive result, as we would assume that a lower pupil teacher ratio would imply a better quality of education (Figure 4 below). When comparing this within the region for primary education, Indonesia performs well amongst its surrounding countries, with only the Philippines being a noticeable outlier to the general sample. In comparison to the developed countries, Indonesia still has some catching up to do but has made some significant improvements, particularly during the period 2006 to 2010. Finally, one area in which Indonesia sees particular progress is in its pupil teacher ratio for secondary education, which shows this as being very strong against the developed countries at an average of 12.4 over the period 2006 to 2010.

Figure 4 Pupil Teacher Ratio comparison

10

Indicators showing challenging trends for Indonesia versus peer countries Indicator 3 Expenditure per student The first indicator identified as being one of potential concern was Expenditure per Student as a % of GDP per capita. For primary and secondary education, Indonesia sits in a relatively low position compared to the surrounding regional countries (albeit in a better position than the Philippines and surprisingly, Singapore). It also performs very poorly against the developed countries and was the worst in this sample. Perhaps more worryingly, there was a downward trend in this indicator between 2009 and 2010 for both primary and secondary education which was in contrast to an upwards trend for both regional and developed countries, though it is difficult to determine whether or not this was only a blip as the data for 2011 and 2012 was not available. Figure 5 summarises some of these observations below for primary education, though the full suite of charts can be found in the Excel Workbook.

Figure 5 Expenditure per student primary education comparisons

Indicator 4 Public Spending as a % GDP In a similar vein to Indicator 3, Indonesia performs in a very similar fashion between both regional countries (i.e. relatively low position) and developed countries (i.e. worst performer) for Public Spending as a % GDP. This is not surprising as these two indicators are very closely linked in that they relate to public spending and the GDP of each country. Figure 6 shows this comparison below:

Figure 6 Public spending as a % GDP comparisons

Indicator 5 Progression to Secondary Education The main thing that jumped out of this indicator was the sudden downward trend between 2006 and 2007 and that at the point of the most recent data for this indicator in 2009, this situation had not improved. In comparison to both the regional and developed countries, Indonesia appears to fare the worst in the sample set. However, it is important that with a very small level of data across 11

a limited timeframe and across few countries, the above conclusions cannot be taken as concrete evidence of Indonesia performing poorly in this indicator, although from the data given, we can assume enough to be concerned. These trends are shown in Figure 7 below:

Figure 7 Progression to Secondary Education % students

Indicator 6 Public Spending as a % Government Expenditure Finally, we identified Public Spending as a % of total Government Expenditure as an interesting trend as this did show Indonesia as the highest ranked compared to the developed countries with a decent performance amongst the regional countries. It was however interesting to note the significant variation in the trend for Indonesia, in particular compared to the developed countries and this posed questions as to how consistent the spending policy on education was or if it was treated as more of a fire-fighting exercise. This could indicate a lack of a long term strategic plan in improving Indonesias educational system as the spending levels are very sporadic (see Figure 8 below).

Figure 8 Public Spending % Government Expenditure

Ranking Tables

We also decided to create ranking tables across each of the indicators to effectively create a balanced scorecard of Indonesias performance ranked against the other sample countries. Each country was ranked within each indicator and within each sample of regional and developed countries. An overall ranking was also created across the 2 sample groups. The ranking tables associated with this analysis can be found in the Excel Workbook with the relevant tab highlighted in Yellow.

12

After considerable debate, it was decided not to draw any firm conclusions from this analysis, as some of the indicators were directly related to the size of the country and therefore not a fair representation of performance. For example, China automatically ranked in 1st place for indicators relating to numbers of teachers and pupils, simply due to the size of the country. Nevertheless, for those indicators that demonstrated a fair comparison of performance, it was still interesting to analyse how Indonesia compared across both sets of countries. Notably: Indonesia ranked in 1st place across both sets of countries for Primary Completion Rate Indonesia also ranked in 1st place for enrolment into primary and secondary private education, however it was important to be aware of what private education meant within the context of each countrys educational system as this could be misleading

Advanced Analysis Inference & Regression

Inference using Normal Distributions

We decided to look at a selection of indicators and determine their distribution across the population of World Bank data. Specifically, we were looking for any instances where a Normal Distribution was present. This would enable us to draw assumptions on the data in terms of Indonesias performance versus the overall population. Five of the indicators were chosen to determine whether such a distribution was present. The relevant sheets and charts relating to this are marked in the Excel Workbook with Blue tabs. The location of Indonesia within each chart was highlighted in red to quickly see its position within the overall population. Following this analysis, it was clear that only 1 of the chosen indicators could be approximated to a normal distribution, namely Public Spending on Education as a % GDP, as can be seen in Figure 9 below:

Figure 9 Normal Distribution of Public Spending on Education as a % GDP

Using this assumption to the Normal Distribution, the following conclusions could be drawn: Mean average = 4.77; Standard Deviation = 1.86; This gave a z score for Indonesia = (2.90-4.77)/1.86 = -1.00 13

Using the Normal Distribution table, the corresponding proportion of the population where public spending as a % GDP was less than that of Indonesia = 15.87% This effectively placed Indonesia in the bottom quartile of the data, with 84.13% of the population of countries spending more money as a % of their GDP on education

Clearly this is a powerful message for the Head of Department and Minister of Education to understand when reviewing educational policy and future spending plans.

Future Projections based on Past Analysis using Regression Techniques

Regression is a technique of determining a relationship between two indicators. Using the graphical pattern of the indicators and significance they had on education, we identified seven pairs of indicators that could potentially have an influence on each other. The analysis for each of these can be found in the Excel workbook under the White tabs. The decision to use linear regression provided us with an opportunity to determine if the data available on the indicators had a direct relationship with each other. We started the analysis by plotting the data in a scatter chart to visually look for trends. Visual analysis was followed by creating a trend line using excel function and extrapolating the correlation coefficient. We used the correlations co-efficient comparison with 2/ and the confidence interval of the slope, not to contain a zero as a check to ensure it there is valuable information provided by performing the regression. We also used the R-Squared value to eliminate pairs due to the data only being able to explain a low proportion of variability. From the seven pairs analysed, two pairs had a relationship where the variance was above 75%. They were: Secondary Education Student Numbers and Secondary Education Teacher Numbers Public Spending % GDP and Public Spending % Govt. Expenditure

Figure 10 Linear Regression relationship between Secondary Education Student Numbers and Secondary Education Teacher Numbers

Figure 10 shows a clear relation between number of students (which we used as the independent variable) and teacher numbers (the dependent variable).

14

Disappointingly, we could not find a clear relationship between Public Spending on Education and Primary Completion Rate or between Expenditure per Student in Primary Education versus Primary Completion rate. This would make it more difficult to find a link between public spending and educational performance. However, for those indicators where a linear relationship was found, the regression modelling provided us with an opportunity of looking at future predictions with confidence intervals. Again, the details of this can be found in the Excel Workbook for the regression analysis (White tabs). This provided us with an opportunity to predict the course of the indicator and make appropriate recommendations to the minister.

Recommendations for Head of Department and Minister of Education

Following analysis of the available data, we would recommend four priority areas for further consideration and appropriate action: 1. Figure 6 demonstrates that Public Spending on Education as a percentage of GDP is consistently lower than many countries in the East Asia region and significantly lower than the developed countries. Furthermore, as outlined in Figure 9, only 16% of worldwide countries tracked by the WDB, report spending less on Education as a percentage of GDP than Indonesia. Over time, it is likely that this underinvestment will weaken the Education framework. To prevent this, the Department of Education must review and update the long term investment strategy for Education. This could be aided by comparing current spend plans to those of developed countries. The Department should also consult with industry to ensure that future employment needs are understood and addressed through future spend plans.

2. As demonstrated by Figure 10, a relationship exists between the number of teachers and pupils. Figure 4 highlights that the pupil teacher ratio has been reducing over the last 10 years, suggesting that class sizes are shrinking, which may increase the quality of learning. The Department of Education should consider the impact of an increasing population and an ageing teaching workforce on this ratio and ensure that a robust teacher training programme is in place, to maintain these class sizes. As new teaching methods evolve, it may be that there is a desire to decrease this ratio further. Through on-going investment in teacher training, Indonesia will be able to maintain and grow the number of qualified teachers. 3. From the Excel Workbook it can be seen that whilst there are comparably high numbers of young people completing primary education, there is a dip in the numbers going onto secondary school. As attendance at secondary school is mandatory for the first three years, it would be expected that the numbers accessing secondary would mirror that of the primary completion level. Some additional qualitative research is necessary to establish why this is the case and depending on the findings, action may need to be taken to encourage parents to send their children to secondary education in Indonesia (as opposed to going abroad for example) and to enforce the compulsory attendance of those young people who find unlawful pathways into work. Increasing the number of young people attending secondary school should have a positive impact on both the level of education in the adult population and the number of young people going onto tertiary education. 15

4. In order to accurately compare Education in Indonesia with other countries and identify trends, good practice and areas for concern, data must be captured and reported more consistently. Whilst there is some very useful data, there were a number of additional indicators which may have been of significant benefit, had the data been available. For example, data in relation to literacy rates of adults can be a good indicator of the quality of primary education. However, this data was not consistently reported. Similarly, whilst the number of young people accessing secondary education was available, there was no data to suggest how many complete and to what level. The Department of Education should give thought to other indicators that may be useful to measure the countrys progress in relation to Education, which can then be used to more accurately direct future strategic planning. Once agreed, monitoring and reporting frameworks can be developed to capture and analyse the information gathered.

16

Part B
The second major part of the exercise involved the use of V.I.S.A. to carry out a Multi Attribute Value Analysis (MAVA) to determine what course of action the Minister of Education should choose when deciding on strategies to improve post-graduate education. The key steps involved in this process are outlined below.

Value Tree
Post Graduate Education was chosen as the main criteria. We then set our first and second tier branches as the following and shown in Figure 11 below: Project Cost o Initial Investment o On-going expenses National Benefit Students o Repatriation rate of Graduates o No. of Candidates o Graduation Rate Institution o Tuition & Living Cost per student o Quality of Education o Prestige of Institution

Figure 11 V.I.S.A. Value Tree

We developed our tier 2 options from a much larger pool of inputs. By role playing as the different stakeholders we were able cut to the core of our concerns. These inputs were then reduced to the final number by going through a mini MCDA exercise and determining which of these were actual concerns within the decision making process, within this context. This allowed us to create our tier 1 groups. It was interesting to note that National Benefit was, on the surface, involved in so many of the inputs that we decided that this had to be a tier 1 input on its own.

17

Scoring
Scoring for each of the items was based on some light research for figures. For those items ranked using High/Medium/Low we used high as the analogy for best i.e. in the case of initial investment, Option A was scored as high as this would have been the cheapest, or best of the alternatives. For the 5 point scoring each alternative was scored in relation to each option. The summary of this scoring can be seen in Figure 12.

Figure 12 Scoring of Criteria

Analysis
Structuring the Problem
We used the CAUSE framework to outline and sanity check that all angles of the problem had been explored. Criteria

From the briefing document it was established that the criteria for this problem was to allow the Minister of Education to evaluate the feasibility of a number of options, or the need for further options for increasing the number of graduates with analytical skills to compliment core subjects.

18

Alternatives

The alternative outcomes analysed were the 3 suggested by the Minister and a 4th best of all worlds option: Option A Send Students to Study Abroad Option B Bring Teaching into Indonesia Option C Develop Courses within Indonesia Option D Combination

Option D was created to capture the best elements of the first 3 options. It was envisaged that a programme would be created in which students were initially sent overseas to study whilst the academic and physical infrastructure for delivering high quality degrees was developed. This development would begin by bringing in high quality academic staff that would eventually be replaced by native staff as the programme matured. Uncertainties

There were many uncertainties discussed from the outset, these uncertainties ranged from either lack of particular knowledge or data such as funding sources, through to variables such as type of courses sought: Funding it was discussed that in all likelihood this would be a combination of government expenditure, NGO funding (such as the ADB) and local fund raising from existing institutions wanting to be involved. Participants it was unknown what the scale of participation would be. Would these programmes only be offered to the academic elite? Would a lottery type system be used? Would such a system be available to all? It was decided that this was an uncertainty which would be clarified better by narrowing the options and so could be discounted until after the feasibility analysis of the current options. Industry involvement it was obvious that Industry would be included in the discussion but what was uncertain was what their role would be. Some of the options discussed included industry as sponsors of students, industry as the beneficiary of higher calibre graduates, or industry as an investor in areas such as research hosted by universities. Again it was decided that even with Industry involvement during the discussion on options, the actual extent of involvement would be better determined once the options had been narrowed down. Type of education although broadly defined by the Minister as more analytical, the types of courses being pursued would have an impact on things like length of study or capital investment. The level of degree (i.e. Doctorate, Masters or Bachelor) was also uncertain.

19

Stakeholders

In the briefing document, the Minister suggested 4 types of stakeholder: Government Industry Local Colleges Students

On discussion it was determined that at this stage it would not be prudent to include local colleges as their vested interests would always bias them. It was discussed if any major stakeholders were missing from this group. During this discussion it was concluded that Government as a stakeholder was too broad a term to encompass the divergent priorities of different areas. This led us to define government stake holders as Government (Education), Government (Finance), and Government (Public Relations). Government Education Government Finance Government Public Relations Industry Students

External / Environmental

Consideration to external influences led us into a broad discussion on culture and graduate opportunities around the world. Although by no means exhaustive, the list of external factors that we felt covered the majority of possible sources of interruption to our decision process were: Availability of international student places in foreign universities. Political relationships with foreign nations. Natural disaster diverting funding. Competition for graduates within host nations or other nations. Change in government through democratic or subversive means. Culture clash leading to non-completion. Game Changer technology leading to decrease in the use of traditional teaching institution.

As mentioned, these factors are not exhaustive but it was felt that these could offer a sufficient threat to the stability of the decision making process that they had to be acknowledged.

20

Weighting
When discussing the weighting for each of the inputs, the group took on the roles of the Stakeholders. The discussion and decision for each of the inputs is outlined below. In order to increase efficiency the weightings were split into tier 1 and tier 2, with tier 2 being discussed first and each grouped according to their parent. Each input in a family was weighted against the other inputs within that family and not within the context of the whole tree. Tier 2

Project Cost o Initial Investment 82.5% o On-going expenses 17.5%

The discussion surround this element was based around the upfront costs of the project. It was agreed that regardless of the option chosen there would be a certain amount of on-going cost. However this could not be discounted completely as there would need to be a balance between how the total cost would be distributed. A figure of $100m for 10 years was suggested as the basis for discussion. This led to discussion on whether $10m every year would return a better investment than say $80m in the first 2 years, then $2.5m each year for the remaining 8 years. The stakeholders had to be reminded that this discussion was to be on the weighting within the decision rather than on the detail of any implementation of any of the options. Students o Repatriation rate of Graduates 7% o No. of Candidates 61% o Graduation Rate 32%

When examining this component the various stakeholders seemed to quickly reach agreement on the repatriation rate very quickly. It was discussed that there was little that could be done to directly affect this trend as international statistics are pretty robust in their stability. A bigger disagreement came about when discussing the number of candidates. Whilst some stake holders i.e. the student, were of the opinion that the pool of potential candidates should be a large consideration, the counter argument was that there would always be candidates available and that if selection criteria were added this would naturally reduce the candidate pool. Agreement was reached between the Industry and Government stakeholders that in considering return on investment, volume was a primary consideration. In regard to graduation rates, it was discussed that with the widening of the criteria in order to increase the pool size, it would be likely that the calibre of the candidates would decrease thus effecting graduation rates. It was decided that it was important to balance the graduation rates against the number of candidates and so was weighted accordingly. 21

Institution o Tuition & living Cost per student 45% o Quality of Education 48% o Prestige of Institution 7%

In discussing the Institution components, all stakeholders agreed that the prestige of the Institutions the graduates were coming from was of minor importance, so long as the quality of the degrees being achieved was sufficiently high, as this would normally correlate with a more prestiges Institute. This then led to a discussion in which the quality of the education was decided as a high weighting factor. The Industry stakeholder argued that they needed quality graduates with good degrees, the government argued that there needed to be a quality component in order to sell the whole project and the student agreed that quality degrees would have more attraction to individual students. The discussion then naturally turned to the cost of quality degrees. As it had been decided that a quality degree would normally be associated with a prestigious institute then there would be a cost premium. It was agreed by all that when looking at individual programs, the cost of them would play a large part when scaling up by numbers and needed to be kept as a major consideration. Tier 1

Once each of the components of the tier 1 inputs was weighted, discussion turned to the relative importance of these and the associated weighting. Involved in this discussion was again the Government with a financial advisor and a PR advisor, Industry representatives and a student body representative. Project Cost 46% National Benefit 35% Students 2% Institution 17%

Discussion began by all parties agreeing that the overall cost of the project was the primary consideration as without the ability to finance, the rest of the inputs were moot. Secondly it was argued by both Industry and the Government that there needed to be national buy in to the program and that there needed to be tangible benefits to the country. At this point Industry stated that its primary consideration was in getting good graduates so they did not really care where they were taught so long as they were of high quality. Government argued that there was a need for internal investment in any case and so each option should be considered with this in mind. In discussing the student input there was general consensus that candidates could be found in any scenario and so in the overall decision making process this was a negligible consideration when weighted against the other inputs. The last input, Institutions, was discussed at some length. Government saw this linking in with National benefit when a home grown option was discussed and that sending Indonesian students to 22

well know institutes around the world would raise the profile of the quality of Indonesian graduates. The student representative also agreed that there should be a non-trivial weighting attached to this in the decision making process. A summary of the weightings applied to each criterion given the above can be seen in Figure 13 below:

Figure 13 Weighting of Criteria

Outcomes
The outcome of the value tree indicated that of the listed alternatives, based on the criteria selected, their scores, and the weighting associated with them, that the most desirable outcome, by a very small margin, was actually option B Bringing in teachers to Indonesia. This can be seen in the score summary charts in Figure 14.

23

Figure 14 Score Summary Charts from V.I.S.A.

By viewing the score profile across all of the options we can clearly see that when each input was taken individually option B came out on top twice, and even then it was joint top with option A in both criteria. If we allocated a score to the position of 4 being top and 1 being bottom we see that option B is again the best outcome but that options C and D are tied, as can be seen in Figure 15.

Figure 15 Alternatives sense check ranking table

24

Weight Sensitivity Analyses

When working through the various criteria there were two critical inputs which showed there would be a dramatic change in outcome if the weighting were to be changed. This first of these, Initial Investment, had from the beginning felt like a critical factor.

Figure 16 Weight sensitivity analysis of Initial Investment vs Post Graduate Education

We can see from Figure 16 that with the exception of option D, the three remaining options all have remarkably steep slopes with all options intersecting at various points. At our agreed weighting we can see that the intersect of option B and option D is extremely close, this is reflected in the overall ranking of the options where we see option D being a very close second place. When taking a step back from this and looking at the tier 1 input, Project Cost, we see this aggressive divergence again emerge, and the same rankings reflected, as shown in Figure 17.

Figure 17 Weight sensitivity analysis of Project Costs vs Post Graduate Education

It is obvious from both common sense and from the evidence seen that project costs are a huge contributory factor in any decision making, and the weighting placed upon it can completely reverse the preferred outcome.

25

The next criterion which was deemed as critical was National Benefit. It was felt that any issue which could impact a nation from top to bottom and would have large cost implications would be a very decisive issue. On running the sensitivity weighting we can clearly see that this is another criterion which, depending on the weighting placed on it by the stakeholders, could radically change the preferred outcomes. Again by looking at the steepness of the slopes we can infer that it would only take a small amount of change in the weighting of the criteria to change the outcome of the decision as can be seen in Figure 18.

Figure 18 Weight sensitivity analysis of National Benefit vs Post Graduate Education

With the current weighting given to this criterion we can see that we are already very close to two cross over points which completely reverse the ranking of the 4 options.

Recommendations
Option B and option D are so close that it is recommended that a second iteration of the process is undertaken covering these as the alternatives, with a further set of criteria developed, in order to examine the root drivers in more depth. Whilst option B gives a definitive course of action, Option D would require a greater amount of prior planning. This additional time in preparing an action plan for Option D may then become the main differentiating factor. If desire for quick implementation is a primary consideration then Option B then becomes the clear outright best fit option. It was interesting to note that Option B was not an option any of the stakeholders were initially biased toward and this could either strengthen the integrity or raise question marks over the scoring and weighting process. This would also increase the argument for a second iteration for the decision making process between Option B and Option D, as the scoring and weighting could be reviewed across the stakeholders again to ensure there was absolute clarity over what was important to each stakeholder.

26

Another conclusion which is important to draw from this exercise is that biased opinion inherently driven by human nature, cannot lead to a purely objective result. Although the importance of including those biased players (as they are the stakeholders) is not in doubt, it is where their input is applied that must be constantly considered and questioned. This emphasises the importance of a facilitator figure to take charge of the process in order to redress any bias which is inserted into the value tree build. This facilitator must objectify as much as possible the reasoning behind any scoring not directly supported by independent fact and also in allocating levels of relative importance. In any case, the exercise acts as a valuable tool to aid the Minister in eliminating options which are not feasible and improves the level of focus on those options that indicate a positive step forwards.

27

References
AngloINFO. (2013). Primary and Secondary Education in Indonesia. Retrieved February 14, 2013, from AngloInfo: the global expat network: http://indonesia.angloinfo.com/family/schoolingeducation/primary-secondary/ Hashtag, T. (2013). ASDM Assignment Excel Workbook. Glasgow. The World Bank Group. (2013). World Bank Data. Retrieved January 20, 2013, from The World Bank: http://data.worldbank.org/topic/education

28

Appendix 1 Task List

Group member Chellappan Thirunavukkarasu

Task PART A

Owen Paterson

Martin Provan

Karen Taylor

Wes beard

Data Cleansing Identifying Gap in Available Data

worked together as group

worked together as group

worked together as group

worked together as group worked together as group worked together as group worked together as group worked together as group Lead discussion of results discussion of results worked together as group

worked together as group worked together as group worked together as group worked together as group worked together as group discussion of results Lead Lead worked together as group

worked together as group

worked together as group

worked together as group

Additional Data Collection Data Preparation & Calculation

worked together as group

worked together as group

worked together as group

worked together as group

worked together as group

worked together as group

Data Visualization Exploratory Analysis Inference Analysis Regression analysis

worked together as group discussion of results discussion of results discussion of results

worked together as group discussion of results discussion of results discussion of results

worked together as group discussion of results discussion of results discussion of results

Recommendations

worked together as group

worked together as group

worked together as group

29

PART B Problem structuring Facilitator role play role play role play worked together as group worked together as group worked together as group Proof read and comments role play worked together as group worked together as group worked together as group Proof read and comments

Scoring and weighting

using software

using software

worked together as group

Analysis

Lead

worked together as group

worked together as group

Recommendation

Lead

worked together as group

worked together as group

Report

Lead

Proof read and comments

Proof read and comments

30