Pig Describe , Explain , Illustrate Operators : Some useful development tools in Pig

Official Website: http://bigdataelearning.com

In this module, you will learn how to use Describe operator, Explain operator and Illustrate operator. These are Pig Latin’s diagnostic operators and using these will you enable to write better code

Describe operator
-----------------------------
Describe operator can be used to view the schema of a relation or alias. For e.g. to view the schema of a alias A, we can give DESCRIBE A;

A = LOAD 'input.txt' USING PigStorage('\t') AS (name:chararray, age:int, salary:int);
DESCRIBE A;
A: {name: chararray,age: int,salary: int}

B = LOAD 'input.txt' USING PigStorage('\t') AS (name, age, salary);
DESCRIBE B;
B: {name: bytearray,age: bytearray,salary: bytearray}

C = LOAD 'input.txt' USING PigStorage('\t');
DESCRIBE C;
Schema for C unknown.

Explain operator
---------------------------
Explain operator is used to display the logical, physical, mapreduce execution plans of an alias or a script. This is helpful to know how pig is compiling each pig commands into mapreduce scripts.

i)The logical plan contains the pipeline of the operators that needs to be executed. It also shows some of the optimizations such as applying filter transformations on the top level are shown.

ii)The physical plan display how the logical operators are converted into backend specific physical operators.

iii)The mapreduce execution plan shows how the physical operators are grouped together to form the mapreduce jobs.

For e.g. to find how these execution plans are laid out, we can type in Explain A;

Illustrate operator
-----------------------------
Illustrate operator is used to display the step by step execution of an alias or a pig script. It is used to debug a code with some small sample of data.

For e.g. Illustrate A, gives step by step execution of the alias A.

If you need to test a script with some small sample of data, then you can give the Illustrate -script PigScript1.pig where PigScript1.pig is the name of the pig script

Видео Pig Describe , Explain , Illustrate Operators : Some useful development tools in Pig канала BigDataElearning