Dremio Fundamentals (learning course); March 18, 2022
The key words are:
- Sources (https://university.dremio.com/courses/course-v1:dremio+D101+2018D101/courseware/f038f3b2912d4dc6ab6e154cc824780a/37dbe6f3033f4578a6ce7e55f930edca/?child=last);
- Physical datasets (https://university.dremio.com/courses/course-v1:dremio+D101+2018D101/courseware/f038f3b2912d4dc6ab6e154cc824780a/43427f08b007402da36e6e1d64253d82/?child=first)
Physical datasets (PDS) are identified by a purple table icon. They have the following distinguishing characteristics:
Physical datasets contain data from a configured source or an uploaded file
Physical datasets are immutable. Data curation and business logic are contained in virtual datasets built on top of physical datasets
Physical datasets can be just one file, or they can be a group of files that share a common schema. Physical datasets can be partitioned in the data source using a directory structure to convey partitioning.
- Spaces (https://university.dremio.com/courses/course-v1:dremio+D101+2018D101/courseware/f038f3b2912d4dc6ab6e154cc824780a/613f42b570964ce093adbdb39da0d41d/?child=first);
Virtual datasets (https://9047-0-2dd1aa07.university-labs.dremio.com/space/Dremio%20Fundamentals);
Virtual datasets are located in shared space or your private home space
Virtual Datasets (VDS) are built on top of one or more physical datasets or other virtual datasets. Virtual datasets have the following important features:
Virtual datasets are defined by an SQL statement that can include filters, joins, unions, type casts, parsing, derived columns, etc. Dremio provides many built-in functions and data curation wizards to provide the functionality needed by the "last mile" of your ETL process.
Virtual datasets are saved in your home space or a shared space
Virtual datasets are a basic building block of your semantic layer. A well-architected semantic layer can greatly maximize both data sharing and data security.
The virtuial datasets are like the "views" of SQL language.