Compilation of AWS DAS certification test points (EMR QuickSight Lakeformation, etc.)

EMR

1. EMR usage scenarios: processing large amounts of data with inconsistent structures.
2. EMR node EBS encryption: LUKS or EBS encryption
3. EMR Hbase high availability: Build additional EMR Hbase read-replica clusters located in different AZs for high availability.
4. EMR master nodes batch initialization: 1.custom bootstrap scripts, 2. AMI
5. EMR master nodes must be in a subnet.
6. Encrypted root device volumn on cluster nodes=customer AMI or security configuration.
7. EMR Auto-Scaling=instance group
8. The permission mechanism for EMR to access S3: Role Mapping. Service Role does not need any S3 permissions. Additional IAM Role can be inherited by Service Role and used by different user groups. (That is, add Service Role to the Trust Policy of each Additional Roles.)
9. Cloudwatch Event + Lambda can start a temporary EMR cluster, and KeepJobFlowAliveWhenNoStep=False will automatically shut down.
10. EMR blocks public network access = account level block public policy
11. Glue invoke EMR = step function

EMRFS

1. EMRFS consistency: 1. Object metadata in DynamoDB, 2. Retry rules.
2. EMRFS does not support SSE-C (S3 Customer Key), but supports KMS key, S3-SSE.
3. EMRFS writes "Slow Down" error: 1. Add Prefix (each preifx in S3 has a list limit, the upper limit is 5000) 2. Add retry for EMRFS
4. EMR list objects slow, increase EMRFS DynamoDB RCU

QuickSight

1. QuickSight is integrated with CloudTrail.
2. Scatter Plot=Determine whether there is a correlation between the two.
3. When QuickSight accesses the newly added S3 Bucket through Athena, a SPICE error is reported: Configure the QuickSight Console to increase the permissions of the S3 Bucket.
4. QuickSight can perform federated queries directly, connecting to Salesforce, mysql and S3.
5. When QuickSight accesses Redshift for the first time, it needs to add the IP to the RD's SG.
6. QuickSight Enterprise Edition has the ML-Powered forecast (forecast widget) function, which can be used when it comes to requiring minimum effort in the algorithm.
7. Enterprise import data limit is 500G, Standard limit is 25GB.
8. QuickSight connects to Redshift across regions: Add QuickSight's IP to RD's SG or VPC Peering+RD managed endpoint via the public network
9. QuickSight cannot directly read Parquet files on S3 but can read JSON, CSV, and XLSX formats.
10. QuickSight Enterprise sharing + permission management = group + folder.

Safety

1. Encryption at rest is only supported by Enterprise edition.
2. QuickSight does not support encryption with customer-provided keys.
3. QuickSight use on-premises AD=AD connector+QuickSight Enterprise edition（AD Connector或SAML2.0）
4. QuickSight Row-level、Column-level security=dataset rules（企业版）。
5. QuickSight multi-tenant, permission management=namespace.

S3 and Lake formation

1. S3 Select supports compression formats (gzip, bzip2), S3 Glacier Select does not support compression formats.
2. Lakeformation Blueprint: Import data from RDS and AWS CloudTrail into S3, and use Glue Crawler directly to build datalake for the data that has entered S3
3. Lakeformation supports cross-account catalog and permission management (IAM+LakeFormation Permission).

Other

OpenSearch 

1. Amazon Opensearch uses IAM for permission management.
2. Amazon Opensearch JVMMemoryPressure problem = too many shards.
3. Amazon Opensearch Ultrawarm Storage & no move back = A quick way to query infrequently used data.

DynamoDB

1. DynamoDB does not support Join.
2. DynamoDB=JSON data，read in millsec.

other

1. Fast Data Curation = DMS+S3+Glue Crawler。
2. Data Exchange Service=share data based on a subscription model。
3. ADF+Amazon Connect (Call Center service) docking, AWS Appflow directly connects to Salesforce and ServiceNow.
4. Amazon Kendra Search Services Integrated with ML
5. SNS Message Filtering function, each subscription can configure Filter Policy.