preface
According to the business needs of the company, you need to upload the folder to oss. After consulting the SDK of oss, you find that it is not supported, so you want to implement it yourself. Because we don't know how deep the folder level is, we can only use recursive processing. As long as it is a file, it will be uploaded after comparing the directory. As long as it is a folder, we will call ourselves and deal with the path level when calling. lz uploads a zip compressed package in the project, and then decompresses it and temporarily stores it in the path specified by the server. It contains the zip decompressed code to see if you need it. Take it yourself, and then use the recursive algorithm to upload all files and directories under the directory. Finally, delete the temporary storage files. Because it is the actual project of the company, there are many other situations to consider, but it's wordy for those who only want to realize the function of uploading folders. If they go straight to the topic, you can check lz's name as the recursionUploadToOSS method,
code implementation
First take a look at the hierarchical path after zip decompression
Related dependency
The following is the dependence of zip decompression and oss. In addition, there are tools for Chinese pinyin conversion. The reason is that there are some transcoding problems when uploading Chinese to oss, so it is directly uploaded after changing to Chinese pinyin
<!--Add compressed package/decompression-->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.18</version>
</dependency>
<!--Alibaba cloud OSS rely on-->
<dependency>
<groupId>com.aliyun.oss</groupId>
<artifactId>aliyun-sdk-oss</artifactId>
<version>3.8.0</version>
</dependency>
<!-- Pinyin tool class library -->
<dependency>
<groupId>com.belerweb</groupId>
<artifactId>pinyin4j</artifactId>
<version>2.5.1</version>
</dependency>
Controller api class
/**
* @param files File array
* @return String
*/
@PostMapping(value = "/batchUploadInteraction")
public String batchUploadInteraction(MultipartFile[] files) {
return interactionDatabaseService.batchUploadInteraction(files, score, difficultyLevel);
}
serviceImpl class
The method is pasted from the written code. Since it is a project code, it will be modified, and we will make our own choices
DEFAULT_OSS_WIN_STORAGE and DEFAULT_OSS_UNITY_BUCKET ,DEFAULT_OSS_UNITY_PATH ,DEFAULT_OSS_LINUX_STORAGE is modified according to its own situation
//The default folder path of interactive animation stored in oss
private static final String DEFAULT_OSS_UNITY_PATH = "certificate/";
//Interactive animation is stored in the default bucket in oss
private static final String DEFAULT_OSS_UNITY_BUCKET = "YOUR BUCKET NAME";
//windows default storage path
private static final String DEFAULT_OSS_WIN_STORAGE = "C:\\Users\\19807\\Desktop\\Doc\\certificate\\";
//Linux default storage path
private static final String DEFAULT_OSS_LINUX_STORAGE = "/data/certificate/temp/";
public String batchUploadInteraction(MultipartFile[] files) {
//Verify the total file size and whether a single file is empty
int totalSize = checkFileTotalSize(files);
//Get current environment
String osName = System.getProperty("os.name");
for (MultipartFile file : files) {
String fileName = file.getOriginalFilename();
int index = fileName.lastIndexOf(".") + 1;
//zip prefix name
String filePrefix = fileName.substring(0, index - 1);
HanYuPinyinHelper hyp = new HanYuPinyinHelper();
//Turn Pinyin
filePrefix = hyp.toHanyuPinyin(filePrefix) + RandomUtil.generateRandom(6);
//file format
String fileSuffix = fileName.substring(index);
if (!FileType.ZIP.getType().equals(fileSuffix)) {
throw new BusinessException(CommonExEnum.FILE_TYPE_ERROR);
}
//zip storage and decompression path
String fileUploadPath;
if (!osName.startsWith("Windows")) {
//linux
fileUploadPath = "/data/certificate/temp/" + filePrefix + "/";
File uploadFile = new File(fileUploadPath);
if (!uploadFile.exists()) {
log.info("Folder directory created:{}", uploadFile.getPath());
uploadFile.mkdirs();
}
} else {
//Windows
fileUploadPath = "C:\\Users\\19807\\Desktop\\Doc\\certificate\\" +
filePrefix + "\\";
File uploadFile = new File(fileUploadPath);
if (!uploadFile.exists()) {
uploadFile.mkdirs();
}
}
//Upload file to server
try {
log.info("try={}", fileUploadPath);
ZipUtil.unzip(file.getInputStream(), fileUploadPath);
} catch (IOException e) {
e.printStackTrace();
}
//Upload all files in this directory to oss according to the hierarchical relationship. Since uploading folders is not supported, all files can only be uploaded recursively
recursionUploadToOSS(fileUploadPath, DEFAULT_OSS_UNITY_PATH);
//Delete temporary files on the server or the development computer, add conditions, and prevent accidental deletion of server files
if (!osName.startsWith("Windows") &&
fileUploadPath.startsWith("/data/certificate/temp/")) {
//linux system
deleteDir(new File(fileUploadPath));
}
}
return "OK";
}
checkFileTotalSize method
It is used to verify whether the file is temperature controlled or the total size exceeds 1024G. If it is not needed, it can be commented directly
/**
* Check whether the single file is empty and the total file size exceeds 1G
*
* @param files Uploaded file array
*/
private int checkFileTotalSize(MultipartFile[] files) {
int totalSize = 0;
for (MultipartFile f : files) {
if (f == null || f.getSize() < 1) {
throw new BusinessException(CommonExEnum.FILE_EMPTY_NONSEXIST);
}
long s = f.getSize() / 1024 / 1024;
log.info("***[{}]File size:{}MB***", f.getOriginalFilename(), s);
totalSize += s;
}
if (totalSize >= 1024) {
throw new BusinessException(CommonExEnum.FILE_SIZE_LIMIT);
}
return totalSize;
}
FileType class
package com.luntek.commons.enums;
/**
* File type, suffix name
*
* @author Czw
* @Date 2020/4/14 0014 10:28 am
*/
public enum FileType {
//zip suffix
ZIP("zip"),
PGN("png"),
MP4("mp4"),
PDF("pdf"),
M3U8("m3u8"),
JPG("jpg");
FileType(String type) {
this.type = type;
}
private String type;
public String getType() {
return type;
}
}
ZipUtil class
package com.luntek.commons.util;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.compress.archivers.ArchiveEntry;
import org.apache.commons.compress.archivers.ArchiveInputStream;
import org.apache.commons.compress.archivers.ArchiveOutputStream;
import org.apache.commons.compress.archivers.zip.ZipArchiveEntry;
import org.apache.commons.compress.archivers.zip.ZipArchiveInputStream;
import org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.springframework.web.bind.annotation.RestController;
import java.io.*;
import java.nio.file.*;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.Objects;
/**
* @author Czw
* @Description Administrator Controller
* @Date 2022/4/18 0017 3:01 PM
*/
@Slf4j
@RestController
public class ZipUtil {
/**
* Compress the folder into the specified output stream, which can be local file output stream or web response download stream
*
* @param srcDir Source folder
* @param outputStream Output stream of compressed file
* @throws IOException IO An exception is thrown to the caller for processing
* @auther CZW
*/
public static void zip(String srcDir, OutputStream outputStream) throws IOException {
try (
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(outputStream);
ArchiveOutputStream out = new ZipArchiveOutputStream(bufferedOutputStream);
) {
Path start = Paths.get(srcDir);
Files.walkFileTree(start, new SimpleFileVisitor<Path>() {
@Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
ArchiveEntry entry = new ZipArchiveEntry(dir.toFile(), start.relativize(dir).toString());
out.putArchiveEntry(entry);
out.closeArchiveEntry();
return super.preVisitDirectory(dir, attrs);
}
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
try (
InputStream input = new FileInputStream(file.toFile())
) {
ArchiveEntry entry = new ZipArchiveEntry(file.toFile(), start.relativize(file).toString());
out.putArchiveEntry(entry);
IOUtils.copy(input, out);
out.closeArchiveEntry();
}
return super.visitFile(file, attrs);
}
});
}
}
/**
* Unzip the zip file to the specified folder
*
* @param zipFileName Source zip file path
* @param destDir Output path after decompression
* @throws IOException IO An exception is thrown to the caller for processing
* @auther CZW
*/
public static void unzip(String zipFileName, String destDir) throws IOException {
try (
InputStream inputStream = new FileInputStream(zipFileName);
) {
unzip(inputStream, destDir);
}
}
/**
* Get the zip file from the input stream and extract it to the specified folder
*
* @param inputStream zip The file input stream can be a local file input stream or a web request upload stream
* @param destDir Output path after decompression
* @throws IOException IO An exception is thrown to the caller for processing
* @auther CZW
*/
public static void unzip(InputStream inputStream, String destDir) throws IOException {
try (
BufferedInputStream bufferedInputStream = new BufferedInputStream(inputStream);
ArchiveInputStream in = new ZipArchiveInputStream(bufferedInputStream);
) {
ArchiveEntry entry;
while (Objects.nonNull(entry = in.getNextEntry())) {
if (in.canReadEntryData(entry)) {
File file = Paths.get(destDir, entry.getName()).toFile();
if (entry.isDirectory()) {
if (!file.exists()) {
file.mkdirs();
}
} else {
try (
OutputStream out = new FileOutputStream(file);
) {
IOUtils.copy(in, out);
}
}
} else {
log.info(entry.getName());
}
}
}
}
}
recursionUploadToOSS method
This method is the core method in the function line of sight, and it also realizes the recursive upload of files
/**
* All files in this directory are uploaded to oss recursively according to the hierarchical relationship. By default, they are uploaded to the bucket of certificate unity exam
*
* @param fileUploadPath Files to be uploaded, for example: C:\certificate\guitichun - cuguizhibei or
* Linux For example: / data/certificate/temp/
* @param obs The path uploaded to oss, for example: certificate/
*/
private void recursionUploadToOSS(String fileUploadPath, String obs) {
File baseFile = new File(fileUploadPath);
//Get all the files in the folder
File[] files = baseFile.listFiles();
for (File file : files) {
//Gets the operation type of the subscript of the path
String osName = System.getProperty("os.name");
int index = osName.startsWith("Windows") ? DEFAULT_OSS_WIN_STORAGE.length() :
DEFAULT_OSS_LINUX_STORAGE.length();
String dir = file.getAbsolutePath().substring(index).
replaceAll("\\\\", "/");
if (file.isDirectory()) {
log.info("[{}]It's a folder", file.getName());
recursionUploadToOSS(file.getAbsolutePath(), obs);
} else if (file.isFile()) {
log.info("[{}]It's a file", file.getName());
//Splicing upload path
String oranFileName = obs + dir;
log.info("***Spliced upload oss The path is:[{}]***", oranFileName);
try {
//upload
OSSUtil.uploadFileWithInputSteam(DEFAULT_OSS_UNITY_BUCKET, oranFileName,
new FileInputStream(file));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
} else {
log.error("[{}]Nothing", file.getName());
}
}
}
deleteDir method
/**
* Recursively delete all files under the directory and all files under the subdirectory
*
* @param dir File directory to be deleted
* @return boolean Returns "true" if all deletions were successful.
* If a deletion fails, the method stops attempting to
* delete and returns "false".
*/
public static boolean deleteDir(File dir) {
if (dir.isDirectory()) {
String[] children = dir.list();
//Recursively delete subdirectories in the directory
for (String child : children) {
boolean success = deleteDir(new File(dir, child));
if (!success) {
return false;
}
}
}
// The directory is empty and can be deleted
return dir.delete();
}
OSSUtil
/**
* Uploading a file stream does not change the file name
*
* @param oranFileName When uploading files to OSS, you need to specify the full path including the file suffix, such as ABC / EFG / 123 jpg
* @param inputStream File stream from local
*/
public static String uploadFileWithInputSteam(String bucketName, String oranFileName, InputStream inputStream) {
// Create an OSSClient instance.
OSS ossClient = new OSSClientBuilder().build(endpoint, accessKeyId, accessKeySecret);
// Upload file stream
try {
//Upload to OSS
ossClient.putObject(bucketName, oranFileName, inputStream);
} catch (Exception ex) {
ex.printStackTrace();
}
// Close the OSSClient.
ossClient.shutdown();
//Returns the full path + name of the file on the server
return getRealName(bucketName, oranFileName);
}
Upload process display
oss directory
log file
Finally, thank you for your review. I hope you will like it and pay attention to the collection. It's not easy to create. It took more than an hour. I could have gone back earlier^_^
The next life is still long. Don't be melancholy