Monthly Archives: July 2014

Realize In-Memory Computation with esProc on Hadoop

Posted on July 31, 2014 by datathinker

The low efficiency of Hadoop computation is an undeniable truth. We believe, one of the major reasons is that the underlying computational structure of MapReduce for Hadoop is basically of the external memory computation. The external memory computation implements the … Continue reading →

Posted in Big Data | Leave a comment

Grouping Function Comparison R Language vs. esProc

Posted on July 30, 2014 by datathinker

Grouping is to allocate the samples into several groups according to a specific flag. There is a difference between groups and the relative commonness shared by group members. The grouping plays an important role in statistical analysis. For example, the … Continue reading →

Posted in FAQ, Program Language | Tagged , , , , | Leave a comment

Implement Column Storage with esProc on Hadoop

Posted on July 30, 2014 by datathinker

The column storage is good, especially when there are lots of tabular fields (this is quite common). In querying, the data to traverse is far less than that on the row storage. Less data to traverse brings less I/O workloads … Continue reading →

Posted in Big Data | Tagged , , , , | Leave a comment

Parallelism of esProc enhances Oracle Data Import Speed

Posted on July 29, 2014 by datathinker

Recently, we skillfully handled an industry project to import a great amount of data from file to Oracle in comparatively short time. At the beginning, we tried to import the data with the sqlldr of Oracle, only to find it … Continue reading →

Posted in Data Analytics | Tagged , , | Leave a comment

esProc Implements Report with File as Data Resource

Posted on July 29, 2014 by datathinker

In the real business, quite a few report data is from file instead of database. Take the below case for example: To compute over the attendance data files for preparing the salary payable report, the log file for preparing the … Continue reading →

Posted in Reporting tool | Tagged , , , , | Leave a comment

ETL or ELT?

Posted on July 28, 2014 by datathinker

ETL process usually includes lot of business logics, which is beyond the reach of visual ETL tools. Most of times they need to be implemented through hard coding. For example, from the production database of a store, we need to … Continue reading →

Posted in Big Data, Data Analytics | Tagged , , , | Leave a comment

Loop Code for esProc

Posted on July 25, 2014 by datathinker

With esProc, the code for loop is mainly implemented with for statement. The for statement will repeat the code block with for as the main cell. There are different formats of the for statements, as listed below: 1.The for loop … Continue reading →

Posted in Unique | Tagged , , , | Leave a comment

Basic Computation of esProc Table Sequence and Record Sequence

Posted on July 24, 2014 by datathinker

esProc table sequence is a structured two-dimensional table, boasting concepts like field, record, primary key and reference. These concepts originate from data table of relational database. A table sequence is also an explicit set of genericity and orderliness, which can … Continue reading →

Posted in Unique | Tagged , , , , | Leave a comment

Basic Computations of esProc Sequences

Posted on July 23, 2014 by datathinker

A sequence is an ordered set consisting of some data, which are called members of the sequence. A sequence is similar to an array in high-level language, but the data type of its members not have to be the same. … Continue reading →

Posted in Unique | Tagged , , | Leave a comment

High efficient development with esProc on Hadoop

Posted on July 22, 2014 by datathinker

Hadoop is an outstanding big data solution. On one hand, its low cost and high scalability increases its popularity; on the other hand, its low development efficiency incurs user complaints. Hadoop is based on the MapReduce framework for big data … Continue reading →

Posted in Big Data | Tagged , , , | Leave a comment