Comparison of Indonesian Population Information System Service Performance Based On Information System Functional Scorecard Theory

The Balikpapan Population and Civil Registration Office has implemented service automation by using an information system to support the population and civil registration administration services in Balikpapan, Indonesia. The information system implemented is called The Population Administration Information System (SIAK). The office has previously implemented an information system similar to SIAK, namely the Population Administration Organizing System (SPAK). One of the office aims is to improve the quality of SIAK. SIAK quality is improved in order to achieve optimal service. As a step of achieving vision and mission, implementing work functions and optimizing the utilization of information systems, the comparison of information system performance between information systems that have been used with information systems that have previously been used. The guidelines used in assessing the performance of both systems are Information System Functional Scorecard (ISFS) theory. The data collected by questionnaires then conducted Paired-T Test so that the results of the two systems were obtained differences and can be compared to know which system is superior. SIAK is an information system that is declared superior to SPAK. Furthermore, the comparison results become a reference in providing recommendations for SIAK performance improvement based on the performance comparison results with SPAK.


Introduction
The utilization of information systems in an organization, especially government organizations, can provide convenience in carrying out its services [1]. After this, the Office of Population and Civil Registration of Balikpapan City, referred to as Dispendukcapil Balikpapan, implements network-based service automation. In 2002, Dispendukcapil Balikpapan developed an information system called the Population Administration System (SPAK) to improve its services. SPAK has a feature that is broadly helpful in serving the entry of civil records of the community and can provide reports on the amount of population data. SPAK also provides receipts or receipts that the community can use as proof of service.
In was conducted en masse, SPAK is no longer used and switched to a new information system issued directly by the central government, namely the Population Administration Information System (SIAK). Based on Indonesian Law 24/2013, SIAK stands for Population Administration Information System that utilizes information and communication technology to facilitate the management of population administration information at the Organizers' and Implementing Agencies as a whole. Services in SIAK Dispendukcapil Balikpapan are population registration, civil registration, archives, registration, online registration, synchronization, and data analysis. With SIAK and e-KTP, data duplication can be reduced because one person cannot have more than one NIK. One of the goals of Dispendukcapil Balikpapan is to improve the quality of SIAK. SIAK quality needs to be improved to achieve optimal service. As a step from achieving the Vision and Mission, implementing the work function, and optimizing the utilization of information systems, it is necessary to know how the comparison of service performance has been used in the Dispendukcapil Balikpapan. An instrument that can be used to measure the performance of an information system so that it can be known the steps to optimize it is the Information System Functional Scrocecard (ISFS). ISFS instruments are comprehensive instruments that have been designed to measure the performance of all information system functions. The instrument measurement is based on a theoretical input-output model of the IS function in supporting business process effectiveness and organizational performance. This instrument consists of three main dimensions: system performance, information effectiveness, and service performance [2]. The method used to determine the difference in SPAK and SIAK is Paired T-Test. Paired T-Test is a test where the data used is not free (paired). The data is contained in one research object and gets two different treatments [3]. Paired T-Test method is considered suitable for use in this study because both systems are contained in the same object, namely Dispendukcapil Balikpapan. This method can also provide information comparing the two systems, with accurate quantitative calculations based on data from ISFS questionnaire results [4].
ISFS was chosen to be the instrument for comparing both systems because based on the previous research conducted by Jerry and William in 2005. This instrument is comprehensive enough to cover all aspects of information system functional performance and sensitive enough to determine the specific area that needs attention.
Research analyzing the quality comparison of two information systems has been conducted previously [13] at Duta Wacana Christian University and STMIK AMIKOM Yogyakarta by measuring the influence of system quality, information quality, and service quality on satisfication users, both together, and independently. The title of this study is "Comparative Analysis of Quality of New Student Admission Services (PMB) Online using The Success Model of DeLone and McLean Information Systems (D&M) (Case Study: PMB UKDW and PMB STMIK AMIKOM Yogyakarta). This type of research is quantitative research using spearman rank correlation analysis techniques and kendall with an ordinal scale. The purpose of this research is to be the basis of the extent of the quality level of PMB Online implemented by STMIK AMIKOM Yogyakarta and Duta Wacana Christian University which is reviewed from user satisfaction in presenting information and public services. From the comparison test, it can be concluded from the user response that the application of the new student admission system conducted online at Duta Wacana Christian University and STMIK AMIKOM Yogyakarta that the success rate of its application is quite good, based on the results of classification of questionnaire data score levels in general has been going well, as well as the success factors of the DeLone and McLean models showed results of 5, which is good. In addition, the results of data processing conducted statistically Descriptive analysis of the validity, reliability, and correlation of ordinal spearman and kendall data is that users assess that the online New Student Admissions (PMB) information system of STMIK AMIKOM Yogyakarta is better than that conducted by The Christian University of Duta Wacana.

Methodology
This study uses a quantitative approach with the research stages in Figure 2.1.

Literature Studies
Literature studies consist of Library studies and Object studies. The library study studied theories about measuring information system performance, previous research discussing similar issues, and the Information System Functional Scorecard (ISFS) instrument. The library's source used is journals, thesis, books, and search results of libraries over the Internet. Object study is conducting preliminary observations on research objects, namely the Civil Registration and Population Office of Balikpapan City.

Preparation for Testing
At this stage, data collection is carried out in a description of the Population Administration Operator System (SPAK) and the Population Administration Information System (SIAK). Then, a questionnaire was prepared to compare the two information system performance under the Information System Functional Scorecard instrument rules.
After the questionnaire was created, the respondents' determination was done using non-probability sampling techniques or random sampling. The type used is Purposive sampling. Validity and reliability must be determined before the questionnaire is ready to collect data from respondents-validity test for a questionnaire using Pearson Moment Product method and Questionnaire reliability test using Cronbach's Alpha equation.

Distribution of Questionnaires
The questionnaires are distributed by providing questionnaires in the form of hardcopy to respondents on the research object. Before completing the questionnaire, respondents were given 5 minutes to read the questions at the beginning of each construct. After that, respondents will be given a certain amount of time to fill out the questionnaire by researchers to ensure accuracy.

Data Analysis
The data obtained will be processed using quantitative calculations under the rules of the Information System Functional Scorecard (ISFS) instrument to obtain information that will then be compared using Paired T-Test. The formula of Paired T-Test was written below in equation 1 [3].

= ̅ √
Where: tcalc = calculated t value ̅ = the average difference between data 1 and data 2 SD = standard deviation of the difference between data 1 and data 2 n = number of samples The test results will determine whether there are differences between the two information system performances that have been implemented and are being implemented.
The research data per domain of each will be calculated in value and then determined by the standard of the value of each questionnaire used as a reference to determine which domain of the system performance is more important to be immediately improved in value. The default values per domain are shown in

SPAK Performance Assessment Questionnaire
Questionnaire-making begins by designing a questionnaire structure divided into three parts. The questionnaire sections are determined based on the domain of ISFS instruments, namely Effectiveness of Information, Information System Service Performance, and System Performance. The first domain has 36 items, the second domain has 26 items, and the third has 26 items.
The first domain's SPAK questionnaire validity test results resulted in as many as ten invalid statement items, the second domain had four invalid statement items, and the third domain had ten invalid items.
Further reliability tests are performed on the details of statements on all three declared valid domains. The reliability test results in all statement items on all three domains being reliable.

SIAK Performance Assessment Questionnaire
SIAK performance assessment questionnaire is created by following the rules of the Information System Functional Scorecard instrument or abbreviated as ISFS. In the first domain, Effectiveness of Information, 36 statements can measure the system effectively on its information. Domain IS Service Performance or Performance Services Information System 26 statement that can measure system information in terms of performance of a given service. This System Performance domain has 38 statements that will measure the process and results of the system. SIAK Questionnaire Validity Test using Pearson Moment Product method with test results showing on the first domain there are twelve items declared invalid, then the second domain there are four items, and the third domain there is one item that is declared invalid. Reliability tests are performed on all three domains that are declared valid-the results of the reliability test state that all items on all three valid domains are declared reliable.

Determination of Respondents
A non-probability purposive sampling technique is used to determine the sample. The method used is Total Population Sampling, where this method will use the entirety of the population members who meet the criteria. This method was chosen because of the limited number of members of the population who meet the criteria. There are only 21 respondents, so that all population members are designated as respondents.
The criteria of respondents are as follows: 1. Employees of the Population and Civil Registration Office of Balikpapan City.

Have experience working using the Population Administration Operator System
(SPAK) and Population Administration Information System (SIAK) for at least three years.

Comparison between SIAK and SPAK Performance
The data was collected from respondents by questionnaires. Data proved to have a normal distribution, then tested into the T-Pair Test to prove which hypothesis can be applied. There are two hypotheses in this SIAK and SPAK system performance comparison study. The hypothesis is described as follows: H0 = There is no difference between SPAK and SIAK performance, so there is no evidence to measure or determine the improvement or decrease in performance of the use of information systems used today compared to previous information systems. H1 = There is a difference between SPAK and SIAK performance to determine whether the information system used today has improved or decreased performance compared to the previous information system.

Paired T-Test
Decision making for data analysis results using the Paired T-Test method from SPSS output is: 1. If sig value (2-tailed) < 0,05, then H0 is successfully rejected.
2. If sig value (2-tailed) > 0,05, then H0 is failed to be rejected. The form of the comparison made using the T-Paired Test is contained in Figure 3.1. In the figure, data from each questionnaire is linked for comparison   successfully rejected. In the previous section, it has also been mentioned that the Sig.
(2-tailed) value of the test result is worth less than 0.05.

Performance Comparison Results Analysis
The previous section conducted the T-Paired Test to determine is H0 rejected or not. In addition, the T-Paired Test can determine the difference between the two systems and which systems are superior based on the scores from the SIAK and SPAK performance assessment questionnaires. The system's superiority is assessed by comparing the number of values between domains. a. Effectiveness of Information The research data collected from the performance assessment questionnaire for the effectiveness domain on the information resulted in the statement that the research hypothesis: "There is a difference between SPAK performance and SIAK, so it can be determined whether the information system used today has improved or decreased performance when compared to the previous information system" accepted. The percentage of SPAK performance comparison value with SIAK from the effectiveness side in the Information contained in Table 3.1 SIAK gets a higher value of 72.35%, while SPAK only gets a value of 66.20%. SIAK is superior in effectiveness to the information produced compared to the information produced by SPAK.

b. IS Service Performance
According to Urbach and Müller (2011) [5], quality of service represents the quality of support users receive from information systems teams and information technology support personnel, such as training provided hotlines or helpdesks. Domain performance of information system services determines the quality assessment of services provided by the information system, named SIAK and SPAK in Dispendukcapil Balikpapan to the people of Balikpapan in the scope of population services and civil registration. The percentage of SPAK performance comparison value with SIAK in the Domain of Information System Service Performance is in Table 3.1 SIAK gets a higher value of 91.78%, while SPAK only gets a value of 81.37%. The results indicate that SIAK again excels in terms of service performance compared to the services provided by SPAK.

c. System Performance
This system performance assessment questionnaire in the System Performance domain has point statements that refer to the assessment of processes and results in the system. In determining the assessment of this domain, the reference that can be used is how the system can access, produce, manipulate, and present information. This domain has six aspects that become a direction in assessing the system's performance. These aspects are Impact on work, Impact on external critical parts, Impact on internal processes, Impact on knowledge and learning, System features, and Ease of use of the system.
The percentage of SPAK with SIAK performance comparison values for System Performance domains is in Table 3.1. SIAK gets a higher value of 62.86%, while SPAK only gets 55.30%. Both values show that SIAK system performance is superior when compared to SPAK.

Conclusions
There is a difference between the performance of the Population Administration Organizing System (SPAK) and the Population Administration Information System (SIAK) to determine whether the information system used today has improved or decreased in terms of performance compared to the previous information system. Furthermore, from the questionnaire data collected, SIAK obtained a higher value than the value obtained by SPAK in the questionnaire domain. Thus, SIAK is superior to SPAK information system in terms of system performance.
Recommendations that can be given to improve SIAK performance is to improve SIAK performance, especially in terms of system performance by paying attention to aspects in the form of system impact for work, external and internal parts, provision of knowledge and learning for system users, as well as improving the quality and variety of system features to support services. The recommended aspect for immediate improvement is in terms of impacting internal processes. Improvements to these aspects can be made by improving management control and improving the quality of the information sharing process from other parts of the organization