You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
437 lines
22 KiB
437 lines
22 KiB
|
1 month ago
|
# 存储管理
|
||
|
|
|
||
|
|
<cite>
|
||
|
|
**本文引用的文件**
|
||
|
|
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
|
||
|
|
- [application.yml](file://crm-app/src/main/resources/application.yml)
|
||
|
|
- [FileApi.java](file://crm-file/src/main/java/com/crm/file/api/FileApi.java)
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [FileConstants.java](file://crm-file/src/main/java/com/crm/file/constant/FileConstants.java)
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileInfoDTO.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [FileDownloadDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileDownloadDTO.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [IFileInfoService.java](file://crm-file/src/main/java/com/crm/file/service/IFileInfoService.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
- [BaseEntity.java](file://crm-base/src/main/java/com/crm/base/domain/entity/BaseEntity.java)
|
||
|
|
- [OwnedEntity.java](file://crm-base/src/main/java/com/crm/base/domain/entity/OwnedEntity.java)
|
||
|
|
- [MybatisPlusConfig.java](file://crm-base/src/main/java/com/crm/base/config/MybatisPlusConfig.java)
|
||
|
|
- [MetaObjectFillHandler.java](file://crm-base/src/main/java/com/crm/base/config/MetaObjectFillHandler.java)
|
||
|
|
- [SnowflakeIdWorker.java](file://crm-base/src/main/java/com/crm/base/config/SnowflakeIdWorker.java)
|
||
|
|
- [SnowflakeProperties.java](file://crm-base/src/main/java/com/crm/base/config/SnowflakeProperties.java)
|
||
|
|
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
|
||
|
|
- [Result.java](file://crm-base/src/main/java/com/crm/base/domain/result/Result.java)
|
||
|
|
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
|
||
|
|
- [BusinessErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/BusinessErrorException.java)
|
||
|
|
</cite>
|
||
|
|
|
||
|
|
## 目录
|
||
|
|
1. [简介](#简介)
|
||
|
|
2. [项目结构](#项目结构)
|
||
|
|
3. [核心组件](#核心组件)
|
||
|
|
4. [架构总览](#架构总览)
|
||
|
|
5. [详细组件分析](#详细组件分析)
|
||
|
|
6. [依赖关系分析](#依赖关系分析)
|
||
|
|
7. [性能与索引优化](#性能与索引优化)
|
||
|
|
8. [配置与容量规划](#配置与容量规划)
|
||
|
|
9. [监控与可观测性](#监控与可观测性)
|
||
|
|
10. [故障排查指南](#故障排查指南)
|
||
|
|
11. [结论](#结论)
|
||
|
|
12. [附录:数据模型与迁移建议](#附录数据模型与迁移建议)
|
||
|
|
|
||
|
|
## 简介
|
||
|
|
本技术文档围绕文件存储管理子系统,系统性阐述 MinIO 对象存储集成、Bucket 管理、权限控制、文件元数据设计、存储路径规划、版本管理、数据库表结构与索引优化、查询性能调优、配置与容量规划、监控方案、生命周期与归档策略、备份恢复机制以及扩展与迁移指南。目标是帮助读者快速理解并高效运维该存储子系统。
|
||
|
|
|
||
|
|
## 项目结构
|
||
|
|
文件存储子系统位于 crm-file 模块,采用分层架构:
|
||
|
|
- API 层:对外暴露统一接口(FileApi)
|
||
|
|
- 控制器层:HTTP 入口(FileController)
|
||
|
|
- 服务层:业务编排与流程控制(IFileInfoService、FileApiImpl、FileInfoServiceImpl)
|
||
|
|
- 客户端层:第三方能力集成(KkFileViewClient)
|
||
|
|
- 配置层:MinIO、调度、属性等(MinioConfig、SchedulingConfig、FileProperties)
|
||
|
|
- 领域层:实体与 DTO(FileInfo、FileInfoDTO、UploadSession 等)
|
||
|
|
- 持久化层:MyBatis-Plus Mapper(FileInfoMapper)
|
||
|
|
- 任务层:定时清理(OrphanChunkCleanupTask)
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
graph TB
|
||
|
|
subgraph "应用启动"
|
||
|
|
App["CrmAppApplication"]
|
||
|
|
end
|
||
|
|
subgraph "文件模块"
|
||
|
|
Ctrl["FileController"]
|
||
|
|
Api["FileApi / FileApiImpl"]
|
||
|
|
Svc["IFileInfoService / FileInfoServiceImpl"]
|
||
|
|
Client["KkFileViewClient"]
|
||
|
|
Cfg["MinioConfig / FileProperties / SchedulingConfig"]
|
||
|
|
Entity["FileInfo / DTOs"]
|
||
|
|
Mapper["FileInfoMapper"]
|
||
|
|
Task["OrphanChunkCleanupTask"]
|
||
|
|
end
|
||
|
|
subgraph "外部系统"
|
||
|
|
MinIO["MinIO 对象存储"]
|
||
|
|
KK["KKFileView 在线预览"]
|
||
|
|
end
|
||
|
|
App --> Ctrl
|
||
|
|
Ctrl --> Api
|
||
|
|
Api --> Svc
|
||
|
|
Svc --> Mapper
|
||
|
|
Svc --> Client
|
||
|
|
Svc --> Cfg
|
||
|
|
Svc --> MinIO
|
||
|
|
Client --> KK
|
||
|
|
Task --> Svc
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApi.java](file://crm-file/src/main/java/com/crm/file/api/FileApi.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [IFileInfoService.java](file://crm-file/src/main/java/com/crm/file/service/IFileInfoService.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileInfoDTO.java)
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [FileDownloadDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileDownloadDTO.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [CrmAppApplication.java](file://crm-app/src/main/java/com/crm/app/CrmAppApplication.java)
|
||
|
|
- [application.yml](file://crm-app/src/main/resources/application.yml)
|
||
|
|
|
||
|
|
## 核心组件
|
||
|
|
- 配置中心
|
||
|
|
- MinioConfig:封装 MinIO 客户端初始化与连接参数
|
||
|
|
- FileProperties:文件相关配置项(如 Bucket、路径前缀、分片大小等)
|
||
|
|
- SchedulingConfig:开启/关闭定时任务(如孤儿分片清理)
|
||
|
|
- 控制器与API
|
||
|
|
- FileController:提供上传、下载、分片上传、预览等 HTTP 接口
|
||
|
|
- FileApi / FileApiImpl:定义并实现文件操作的业务契约
|
||
|
|
- 服务层
|
||
|
|
- IFileInfoService / FileInfoServiceImpl:文件元数据管理、上传流程编排、下载流程、预览链接生成
|
||
|
|
- KkFileViewClient:调用第三方在线预览服务
|
||
|
|
- 持久化
|
||
|
|
- FileInfo 实体与 FileInfoMapper:文件元数据持久化
|
||
|
|
- 任务
|
||
|
|
- OrphanChunkCleanupTask:定期清理未完成的分片上传残留
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApi.java](file://crm-file/src/main/java/com/crm/file/api/FileApi.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [IFileInfoService.java](file://crm-file/src/main/java/com/crm/file/service/IFileInfoService.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
|
||
|
|
## 架构总览
|
||
|
|
整体采用“控制器-服务-客户端”的分层模式,结合 MinIO 对象存储与第三方预览服务,形成高内聚、低耦合的文件管理能力。
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
sequenceDiagram
|
||
|
|
participant Client as "客户端"
|
||
|
|
participant Controller as "FileController"
|
||
|
|
participant Service as "IFileInfoService/FileInfoServiceImpl"
|
||
|
|
participant MinIO as "MinIO 对象存储"
|
||
|
|
participant KK as "KKFileView 预览服务"
|
||
|
|
Client->>Controller : "发起上传/下载/预览请求"
|
||
|
|
Controller->>Service : "校验参数与权限"
|
||
|
|
alt 上传流程
|
||
|
|
Service->>MinIO : "创建Bucket(若不存在)/生成对象键"
|
||
|
|
Service->>MinIO : "分片上传或直传"
|
||
|
|
Service-->>Controller : "返回文件元数据"
|
||
|
|
else 下载流程
|
||
|
|
Service->>MinIO : "获取对象流/签名URL"
|
||
|
|
Service-->>Controller : "返回下载响应"
|
||
|
|
else 预览流程
|
||
|
|
Service->>KK : "生成预览链接"
|
||
|
|
Service-->>Controller : "返回预览地址"
|
||
|
|
end
|
||
|
|
Controller-->>Client : "统一结果封装"
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
|
||
|
|
## 详细组件分析
|
||
|
|
|
||
|
|
### MinIO 对象存储集成
|
||
|
|
- 目标:通过 MinioConfig 初始化客户端,使用 FileProperties 中的 Bucket、Endpoint、AccessKey、SecretKey 等配置进行连接与鉴权。
|
||
|
|
- 关键职责:
|
||
|
|
- 动态创建/检查 Bucket 存在性
|
||
|
|
- 生成对象键(路径规划)
|
||
|
|
- 上传(直传/分片)、下载(流式/签名URL)、删除、元数据读取
|
||
|
|
- 权限控制:
|
||
|
|
- 基于访问密钥的细粒度权限
|
||
|
|
- 可在 MinIO 控制台设置 Bucket 策略,限制读写范围
|
||
|
|
- 路径规划建议:
|
||
|
|
- 按租户/业务域划分 Bucket
|
||
|
|
- 对象键采用“业务类型/日期/唯一标识”三段式,便于检索与归档
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
|
||
|
|
### Bucket 管理
|
||
|
|
- 自动创建:首次写入时检测 Bucket 是否存在,不存在则创建
|
||
|
|
- 命名规范:建议使用小写英文字母与数字,避免冲突
|
||
|
|
- 生命周期策略:在 MinIO 侧配置规则(过期、转储到冷存储)
|
||
|
|
- 多环境隔离:开发/测试/生产分别独立 Bucket
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
|
||
|
|
### 权限控制
|
||
|
|
- 认证与授权:由上层安全模块负责(JWT、角色、数据权限),文件接口需鉴权后访问
|
||
|
|
- 数据可见性:通过 OwnedEntity 基类注入所有者信息,确保仅本人或授权角色可操作
|
||
|
|
- 最小权限原则:为 MinIO 访问密钥配置最小可用权限(只读/只写特定 Bucket)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [OwnedEntity.java](file://crm-base/src/main/java/com/crm/base/domain/entity/OwnedEntity.java)
|
||
|
|
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
|
||
|
|
- [Result.java](file://crm-base/src/main/java/com/crm/base/domain/result/Result.java)
|
||
|
|
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
|
||
|
|
|
||
|
|
### 文件元数据设计与存储路径规划
|
||
|
|
- 元数据字段建议:
|
||
|
|
- 唯一标识、文件名、原始大小、MIME 类型、MD5/SHA1、存储桶名、对象键、上传者、所属者、状态、版本、创建/更新时间
|
||
|
|
- 路径规划:
|
||
|
|
- 示例:/{tenant}/{bizType}/{yyyyMMdd}/{uuid}.{ext}
|
||
|
|
- 支持按业务域与时间分区,提升检索与归档效率
|
||
|
|
- 版本管理:
|
||
|
|
- 建议在对象键中追加版本号或使用 MinIO 版本控制功能
|
||
|
|
- 元数据记录当前版本与历史版本映射
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileInfoDTO.java)
|
||
|
|
- [FileConstants.java](file://crm-file/src/main/java/com/crm/file/constant/FileConstants.java)
|
||
|
|
|
||
|
|
### 分片上传与会话管理
|
||
|
|
- 流程:
|
||
|
|
- 初始化会话(MultipartInitDTO)
|
||
|
|
- 逐块上传(UploadSession 维护分片状态)
|
||
|
|
- 合并分片并落盘元数据
|
||
|
|
- 异常处理:
|
||
|
|
- 网络中断重试、分片完整性校验
|
||
|
|
- 超时与幂等控制
|
||
|
|
- 清理策略:
|
||
|
|
- OrphanChunkCleanupTask 定期扫描并清理未完成会话
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [MultipartInitDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/MultipartInitDTO.java)
|
||
|
|
- [UploadSession.java](file://crm-file/src/main/java/com/crm/file/domain/dto/UploadSession.java)
|
||
|
|
- [OrphanChunkCleanupTask.java](file://crm-file/src/main/java/com/crm/file/task/OrphanChunkCleanupTask.java)
|
||
|
|
|
||
|
|
### 在线预览集成
|
||
|
|
- 通过 KkFileViewClient 调用第三方预览服务,返回预览 URL
|
||
|
|
- 支持常见办公文档格式,无需本地转换
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
|
||
|
|
### 下载流程
|
||
|
|
- 支持直接流式下载与签名 URL 下载
|
||
|
|
- 根据文件大小与场景选择合适方式
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileDownloadDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileDownloadDTO.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
|
||
|
|
## 依赖关系分析
|
||
|
|
- 模块内依赖:
|
||
|
|
- FileController -> FileApi -> IFileInfoService -> FileInfoMapper
|
||
|
|
- FileApiImpl/FileInfoServiceImpl -> MinioConfig/FileProperties
|
||
|
|
- FileInfoServiceImpl -> KkFileViewClient
|
||
|
|
- 基础依赖:
|
||
|
|
- MyBatis-Plus 配置(MybatisPlusConfig、MetaObjectFillHandler、SnowflakeIdWorker)
|
||
|
|
- 全局异常处理与统一结果封装
|
||
|
|
|
||
|
|
```mermaid
|
||
|
|
classDiagram
|
||
|
|
class FileController
|
||
|
|
class FileApi
|
||
|
|
class FileApiImpl
|
||
|
|
class IFileInfoService
|
||
|
|
class FileInfoServiceImpl
|
||
|
|
class FileInfoMapper
|
||
|
|
class MinioConfig
|
||
|
|
class FileProperties
|
||
|
|
class KkFileViewClient
|
||
|
|
class FileInfo
|
||
|
|
class FileInfoDTO
|
||
|
|
FileController --> FileApi : "调用"
|
||
|
|
FileApi <|.. FileApiImpl : "实现"
|
||
|
|
FileApiImpl --> IFileInfoService : "委托"
|
||
|
|
IFileInfoService <|.. FileInfoServiceImpl : "实现"
|
||
|
|
FileInfoServiceImpl --> FileInfoMapper : "持久化"
|
||
|
|
FileInfoServiceImpl --> MinioConfig : "配置"
|
||
|
|
FileInfoServiceImpl --> FileProperties : "配置"
|
||
|
|
FileInfoServiceImpl --> KkFileViewClient : "预览"
|
||
|
|
FileInfoServiceImpl --> FileInfo : "实体"
|
||
|
|
FileInfoServiceImpl --> FileInfoDTO : "数据传输"
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileController.java](file://crm-file/src/main/java/com/crm/file/controller/FileController.java)
|
||
|
|
- [FileApi.java](file://crm-file/src/main/java/com/crm/file/api/FileApi.java)
|
||
|
|
- [FileApiImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileApiImpl.java)
|
||
|
|
- [IFileInfoService.java](file://crm-file/src/main/java/com/crm/file/service/IFileInfoService.java)
|
||
|
|
- [FileInfoServiceImpl.java](file://crm-file/src/main/java/com/crm/file/service/impl/FileInfoServiceImpl.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [MinioConfig.java](file://crm-file/src/main/java/com/crm/file/config/MinioConfig.java)
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [KkFileViewClient.java](file://crm-file/src/main/java/com/crm/file/service/KkFileViewClient.java)
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoDTO.java](file://crm-file/src/main/java/com/crm/file/domain/dto/FileInfoDTO.java)
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [MybatisPlusConfig.java](file://crm-base/src/main/java/com/crm/base/config/MybatisPlusConfig.java)
|
||
|
|
- [MetaObjectFillHandler.java](file://crm-base/src/main/java/com/crm/base/config/MetaObjectFillHandler.java)
|
||
|
|
- [SnowflakeIdWorker.java](file://crm-base/src/main/java/com/crm/base/config/SnowflakeIdWorker.java)
|
||
|
|
- [SnowflakeProperties.java](file://crm-base/src/main/java/com/crm/base/config/SnowflakeProperties.java)
|
||
|
|
|
||
|
|
## 性能与索引优化
|
||
|
|
- 数据库索引建议:
|
||
|
|
- 主键:雪花ID(SnowflakeIdWorker)
|
||
|
|
- 常用查询列:所属者、业务类型、创建时间、状态、MD5
|
||
|
|
- 复合索引:(owner_id, biz_type, create_time)
|
||
|
|
- 唯一索引:md5(去重校验)
|
||
|
|
- 查询优化:
|
||
|
|
- 分页查询使用合理页大小
|
||
|
|
- 避免 SELECT *,按需投影
|
||
|
|
- 大对象不入库,仅存元数据
|
||
|
|
- 上传优化:
|
||
|
|
- 分片大小与并发数调优
|
||
|
|
- 启用断点续传与重试
|
||
|
|
- 缓存策略:
|
||
|
|
- 热点文件元数据可缓存(Redis)
|
||
|
|
- 预览 URL 短期缓存
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [SnowflakeIdWorker.java](file://crm-base/src/main/java/com/crm/base/config/SnowflakeIdWorker.java)
|
||
|
|
- [SnowflakeProperties.java](file://crm-base/src/main/java/com/crm/base/config/SnowflakeProperties.java)
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
|
||
|
|
## 配置与容量规划
|
||
|
|
- 配置项建议:
|
||
|
|
- MinIO Endpoint、AccessKey、SecretKey、Bucket、Region
|
||
|
|
- 分片大小、最大分片数、超时时间
|
||
|
|
- 是否启用预览服务、预览服务地址
|
||
|
|
- 定时任务开关与执行周期
|
||
|
|
- 容量规划:
|
||
|
|
- 预估年增量与峰值吞吐
|
||
|
|
- 冷热数据分离策略(热数据 SSD,冷数据 HDD/归档)
|
||
|
|
- 副本与纠删码策略(MinIO 默认纠删码)
|
||
|
|
- 生命周期与归档:
|
||
|
|
- 90 天未访问转冷存储
|
||
|
|
- 180 天归档至低成本介质
|
||
|
|
- 365 天合规删除
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileProperties.java](file://crm-file/src/main/java/com/crm/file/config/FileProperties.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|
||
|
|
|
||
|
|
## 监控与可观测性
|
||
|
|
- 指标采集:
|
||
|
|
- MinIO 指标(QPS、延迟、错误率、磁盘使用)
|
||
|
|
- 应用指标(上传成功率、分片失败率、预览成功率)
|
||
|
|
- 日志规范:
|
||
|
|
- 统一 TraceId 贯穿请求链路
|
||
|
|
- 关键步骤打点(初始化、上传、合并、下载、预览)
|
||
|
|
- 告警策略:
|
||
|
|
- 上传失败率阈值
|
||
|
|
- 存储空间使用率阈值
|
||
|
|
- 预览服务不可用告警
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [application.yml](file://crm-app/src/main/resources/application.yml)
|
||
|
|
|
||
|
|
## 故障排查指南
|
||
|
|
- 常见问题定位:
|
||
|
|
- 连接失败:检查 MinIO 配置与网络连通性
|
||
|
|
- 权限不足:核对访问密钥权限与 Bucket 策略
|
||
|
|
- 分片上传失败:检查分片大小、超时与重试策略
|
||
|
|
- 预览失败:确认预览服务可达与文件格式支持
|
||
|
|
- 工具与手段:
|
||
|
|
- 查看应用日志与 MinIO 审计日志
|
||
|
|
- 使用 MinIO 控制台验证对象存在性与元数据
|
||
|
|
- 通过统一结果封装与全局异常处理器定位错误码
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [GlobalExceptionHandlerAdvice.java](file://crm-base/src/main/java/com/crm/base/advice/GlobalExceptionHandlerAdvice.java)
|
||
|
|
- [Result.java](file://crm-base/src/main/java/com/crm/base/domain/result/Result.java)
|
||
|
|
- [ResultCodeEnum.java](file://crm-base/src/main/java/com/crm/base/domain/result/ResultCodeEnum.java)
|
||
|
|
- [BusinessErrorException.java](file://crm-base/src/main/java/com/crm/base/domain/exception/BusinessErrorException.java)
|
||
|
|
|
||
|
|
## 结论
|
||
|
|
本存储管理子系统以 MinIO 为核心,结合完善的元数据管理与第三方预览能力,提供了稳定高效的文件处理能力。通过合理的索引与配置、完善的监控与告警、清晰的生命周期与归档策略,可满足企业级文件存储需求。后续可按业务增长进行水平扩展与迁移升级。
|
||
|
|
|
||
|
|
## 附录:数据模型与迁移建议
|
||
|
|
|
||
|
|
### 数据模型(FileInfo 实体)
|
||
|
|
```mermaid
|
||
|
|
erDiagram
|
||
|
|
FILE_INFO {
|
||
|
|
bigint id PK
|
||
|
|
string file_name
|
||
|
|
bigint size
|
||
|
|
string mime_type
|
||
|
|
string md5
|
||
|
|
string bucket
|
||
|
|
string object_key
|
||
|
|
bigint owner_id
|
||
|
|
int status
|
||
|
|
int version
|
||
|
|
datetime created_at
|
||
|
|
datetime updated_at
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
图表来源
|
||
|
|
- [FileInfo.java](file://crm-file/src/main/java/com/crm/file/domain/entity/FileInfo.java)
|
||
|
|
- [BaseEntity.java](file://crm-base/src/main/java/com/crm/base/domain/entity/BaseEntity.java)
|
||
|
|
- [OwnedEntity.java](file://crm-base/src/main/java/com/crm/base/domain/entity/OwnedEntity.java)
|
||
|
|
|
||
|
|
### 迁移与扩展建议
|
||
|
|
- 横向扩展:
|
||
|
|
- 多实例部署共享 MinIO,无状态化文件服务
|
||
|
|
- 引入消息队列异步处理预览与转码
|
||
|
|
- 迁移方案:
|
||
|
|
- 使用 MinIO 官方迁移工具批量迁移历史数据
|
||
|
|
- 双写过渡期保证一致性,逐步切换流量
|
||
|
|
- 版本管理:
|
||
|
|
- 启用 MinIO 版本控制,保留历史版本
|
||
|
|
- 元数据维护版本映射与回滚策略
|
||
|
|
|
||
|
|
章节来源
|
||
|
|
- [FileInfoMapper.java](file://crm-file/src/main/java/com/crm/file/mapper/FileInfoMapper.java)
|
||
|
|
- [SchedulingConfig.java](file://crm-file/src/main/java/com/crm/file/config/SchedulingConfig.java)
|