I'm developing a Lambda function with two handlers: one for scraping websites, and one for building a static website based on the scraped and combined content.
I have created two separate classes with methods two handle both the scraping and the website building. Both classes use the AWS Java SDK to access services such as S3 and DynamoDB. I am also using other libraries such as Apache Free Marker, Gson, etc.
I have ensured that in my code, only specific classes are imported and avoided any import .* type expressions.
After I build and deploy my Lambda, and inspect the deployment package, I notice the enormous size: 123 megabyte! Reason for the large package size is that a whopping 211 (!) jar files are included in the lib folder.
When I further analyse the included jar files, I noticed that 90%+ of the jar's are from the AWS SDK.
My question: is this a normal file size for a Lambda deployment package when building apps with integrations to different AWS services, or am I doing something wrong here? In my view this is not a complex app at all and the amount of jar's that's been imported seems excessive and blowing up the deployment package.
Edit 1: my pom.xml is included below:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>helloworld</groupId>
<artifactId>HelloWorld</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<name>A sample Hello World created for SAM CLI.</name>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencyManagement>
<dependencies>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bom -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-bom</artifactId>
<version>1.11.651</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
<version>1.2.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<!-- jsoup HTML parser library @ https://jsoup.org/ -->
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.12.1</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-lambda-java-events -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-events</artifactId>
<version>2.2.7</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.659</version>
</dependency>
<!-- https://mvnrepository.com/artifact/software.amazon.awssdk/sdk-core -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<!--<version>1.11.651</version>-->
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-dynamodb</artifactId>
<!--<version>1.11.651</version>-->
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.code.gson/gson -->
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3 -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<!--<version>1.11.651</version>-->
</dependency>
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
<version>2.3.29</version>
</dependency>
<!-- https://mvnrepository.com/artifact/commons-io/commons-io -->
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.6</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.1</version>
<configuration>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Edit 2: after excluding the 'aws-java-sdk' dependency and re-building the project, I was able to run the 'Deploy Serverless Application' command again, and it worked. The deployment package size is down to 9.3 megabyte and 21 jar's - I checked the included jar's and they're in fact those I actually need.
The problem here is that you have the following as one of your dependencies:
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.659</version>
</dependency>
This results in your project depending on the entire AWS Java SDK, instead of individual service modules.
The solution is to remove this dependency, and only depend on the modules that are needed for your project to function properly.