Java GenericObjectPool 物件池化技術--SpringBoot sftp 連線池工具類
2023-04-07 09:00:24
Java BasePooledObjectFactory 物件池化技術
通常一個物件建立、銷燬非常耗時的時候，我們不會頻繁的建立和銷燬它，而是考慮複用。複用物件的一種做法就是物件池，將建立好的物件放入池中維護起來，下次再用的時候直接拿池中已經建立好的物件繼續用，這就是池化的思想。
Apache Commons Pool是一個物件池的框架，他提供了一整套用於實現物件池化的API。它提供了三種物件池：GenericKeyedObjectPool，SoftReferenceObjectPool和GenericObjectPool，其中GenericObjectPool是我們最常用的物件池，內部實現也最複雜。
GenericObjectPool
GenericObjectPool 是一個通用物件池框架，我們可以藉助它實現一個健壯的物件池，UML圖如下所示：
GenericObjectPool 實現了ObjectPool介面，而ObjectPool就是物件池的核心介面，它定義了一個物件池應該實現的行為。
public interface ObjectPool<T> extends Closeable {
/**
* 從池中借走到一個物件
*/
T borrowObject() throws Exception, NoSuchElementException, IllegalStateException;
/**
* 把物件歸還給物件池
*/
void returnObject(T var1) throws Exception;
/**
* 驗證物件的有效性
*/
void invalidateObject(T var1) throws Exception;
/**
* 往池中新增一個物件
*/
void addObject() throws Exception, IllegalStateException, UnsupportedOperationException;
/**
* 返回物件池中有多少物件是空閒的，也就是能夠被借走的物件的數量。
*/
int getNumIdle();
/**
* 返回物件池中有物件物件是活躍的，也就是已經被借走的，在使用中的物件的數量。
*/
int getNumActive();
/**
* 清理物件池。注意是清理不是清空，該方法要求的是，清理所有空閒物件，釋放相關資源。
*/
void clear() throws Exception, UnsupportedOperationException;
/**
* 關閉物件池。這個方法可以達到清空的效果，清理所有物件以及相關資源。
*/
void close();
}
BasePooledObjectFactory
Java BasePooledObjectFactory 物件池化技術
使用GenericObjectPool只需要建立一個物件工廠類，繼承BasePooledObjectFactory並重寫它的create()和destroyObject()。
如下文中的：SftpPool.java
public interface PooledObjectFactory<T> {
/**
* 建立一個可由池提供服務的範例，並將其封裝在由池管理的PooledObject中。
*/
PooledObject<T> makeObject() throws Exception;
/**
* 銷燬池不再需要的範例
*/
void destroyObject(PooledObject<T> var1) throws Exception;
/**
* 確保範例可以安全地由池返回
*/
boolean validateObject(PooledObject<T> var1);
/**
* 重新初始化池返回的範例
*/
void activateObject(PooledObject<T> var1) throws Exception;
/**
* 取消初始化要返回到空閒物件池的範例
*/
void passivateObject(PooledObject<T> var1) throws Exception;
}
設定類GenericObjectPoolConfig
GenericObjectPoolConfig是封裝GenericObject池設定的簡單「結構」，此類不是執行緒安全的；它僅用於提供建立池時使用的屬性。大多數情況，可以使用GenericObjectPoolConfig提供的預設引數就可以滿足日常的需求。
工作原理流程
- 構造方法
當我們執行構造方法時，主要工作就是建立了一個儲存物件的LinkedList型別容器，也就是概念意義上的「池」 - 從物件池中獲取物件
獲取池中的物件是通過borrowObject()命令，原始碼比較複雜，簡單而言就是去LinkedList中獲取一個物件，如果不存在的話，要呼叫構造方法中第一個引數Factory工廠類的makeObject()方法去建立一個物件再獲取，獲取到物件後要呼叫validateObject方法判斷該物件是否是可用的，如果是可用的才拿去使用。LinkedList容器減一 - 歸還物件到執行緒池
簡單而言就是先呼叫validateObject方法判斷該物件是否是可用的，如果可用則歸還到池中，LinkedList容器加一，如果是不可以的則呼叫destroyObject方法進行銷燬
上面三步就是最簡單的流程，由於取和還的流程步驟都在borrowObject和returnObject方法中固定的，所以我們只要重寫Factory工廠類的makeObject()和validateObject以及destroyObject方法即可實現最簡單的池的管理控制，通過構造方法傳入該Factory工廠類物件則可以建立最簡單的物件池管理類。這算是比較好的解耦設計模式，借和還的流程如下圖所示：
使用Demo
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<version>2.7.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.55</version>
</dependency>
點選檢視程式碼
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>vipsoft-parent</artifactId>
<groupId>com.vipsoft.boot</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>vipsoft-sftp</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-pool2</artifactId>
<version>2.7.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
<dependency>
<groupId>com.jcraft</groupId>
<artifactId>jsch</artifactId>
<version>0.1.55</version>
</dependency>
<dependency>
<groupId>org.eclipse.paho</groupId>
<artifactId>org.eclipse.paho.client.mqttv3</artifactId>
<version>1.2.5</version>
</dependency>
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>5.3.6</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
application.yaml
server:
port: 8088
application:
name: sftp Demo
sftp:
host: 172.16.3.88 # 伺服器ip
port: 22 # ssh埠
username: root # 使用者名稱
password: root # 密碼
# 連線池引數
pool:
max-total: 10
max-idle: 10
min-idle: 5
SftpPoolException.java
package com.vipsoft.sftp.exception;
/**
* sftp連線池異常
*/
public class SftpPoolException extends RuntimeException {
private static final long serialVersionUID = 1L;
/**
* Constructs a new runtime exception with {@code null} as its
* detail message. The cause is not initialized, and may subsequently be
* initialized by a call to {@link #initCause}.
*/
public SftpPoolException() {
}
/**
* Constructs a new runtime exception with the specified detail message.
* The cause is not initialized, and may subsequently be initialized by a
* call to {@link #initCause}.
*
* @param message the detail message. The detail message is saved for
* later retrieval by the {@link #getMessage()} method.
*/
public SftpPoolException(String message) {
super(message);
}
/**
* Constructs a new runtime exception with the specified detail message and
* cause. <p>Note that the detail message associated with
* {@code cause} is <i>not</i> automatically incorporated in
* this runtime exception's detail message.
*
* @param message the detail message (which is saved for later retrieval
* by the {@link #getMessage()} method).
* @param cause the cause (which is saved for later retrieval by the
* {@link #getCause()} method). (A <tt>null</tt> value is
* permitted, and indicates that the cause is nonexistent or
* unknown.)
* @since 1.4
*/
public SftpPoolException(String message, Throwable cause) {
super(message, cause);
}
/**
* Constructs a new runtime exception with the specified cause and a
* detail message of <tt>(cause==null ? null : cause.toString())</tt>
* (which typically contains the class and detail message of
* <tt>cause</tt>). This constructor is useful for runtime exceptions
* that are little more than wrappers for other throwables.
*
* @param cause the cause (which is saved for later retrieval by the
* {@link #getCause()} method). (A <tt>null</tt> value is
* permitted, and indicates that the cause is nonexistent or
* unknown.)
* @since 1.4
*/
public SftpPoolException(Throwable cause) {
super(cause);
}
/**
* Constructs a new runtime exception with the specified detail
* message, cause, suppression enabled or disabled, and writable
* stack trace enabled or disabled.
*
* @param message the detail message.
* @param cause the cause. (A {@code null} value is permitted,
* and indicates that the cause is nonexistent or unknown.)
* @param enableSuppression whether or not suppression is enabled
* or disabled
* @param writableStackTrace whether or not the stack trace should
* be writable
* @since 1.7
*/
public SftpPoolException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
super(message, cause, enableSuppression, writableStackTrace);
}
}
config
SftpConfig.java
package com.vipsoft.sftp.config;
import com.vipsoft.sftp.pool.SftpFactory;
import com.vipsoft.sftp.pool.SftpPool;
import com.vipsoft.sftp.utils.SftpUtil;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableConfigurationProperties(SftpProperties.class)
public class SftpConfig {
// 工廠
@Bean
public SftpFactory sftpFactory(SftpProperties properties) {
return new SftpFactory(properties);
}
// 連線池
@Bean
public SftpPool sftpPool(SftpFactory sftpFactory) {
return new SftpPool(sftpFactory);
}
// 輔助類
@Bean
public SftpUtil sftpUtil(SftpPool sftpPool) {
return new SftpUtil(sftpPool);
}
}
SftpProperties.java
package com.vipsoft.sftp.config;
import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix = "sftp")
public class SftpProperties {
private String host;
private int port = 22;
private String username = "root";
private String password = "root";
private Pool pool = new Pool();
public String getHost() {
return host;
}
public void setHost(String host) {
this.host = host;
}
public int getPort() {
return port;
}
public void setPort(int port) {
this.port = port;
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public String getPassword() {
return password;
}
public void setPassword(String password) {
this.password = password;
}
public Pool getPool() {
return pool;
}
public void setPool(Pool pool) {
this.pool = pool;
}
public static class Pool extends GenericObjectPoolConfig<ChannelSftp> {
private int maxTotal = DEFAULT_MAX_TOTAL;
private int maxIdle = DEFAULT_MAX_IDLE;
private int minIdle = DEFAULT_MIN_IDLE;
public Pool() {
super();
}
@Override
public int getMaxTotal() {
return maxTotal;
}
@Override
public void setMaxTotal(int maxTotal) {
this.maxTotal = maxTotal;
}
@Override
public int getMaxIdle() {
return maxIdle;
}
@Override
public void setMaxIdle(int maxIdle) {
this.maxIdle = maxIdle;
}
@Override
public int getMinIdle() {
return minIdle;
}
@Override
public void setMinIdle(int minIdle) {
this.minIdle = minIdle;
}
}
}
Pool
SftpFactory.java
package com.vipsoft.sftp.pool;
import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.vipsoft.sftp.config.SftpProperties;
import com.vipsoft.sftp.exception.SftpPoolException;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Properties;
public class SftpFactory extends BasePooledObjectFactory<ChannelSftp> {
private final Logger logger = LoggerFactory.getLogger(this.getClass());
private SftpProperties properties;
public SftpProperties getProperties() {
return properties;
}
public void setProperties(SftpProperties properties) {
this.properties = properties;
}
public SftpFactory(SftpProperties properties) {
this.properties = properties;
}
@Override
public ChannelSftp create() {
try {
JSch jsch = new JSch();
Session sshSession = jsch.getSession(properties.getUsername(), properties.getHost(), properties.getPort());
sshSession.setPassword(properties.getPassword());
Properties sshConfig = new Properties();
sshConfig.put("StrictHostKeyChecking", "no");
sshSession.setConfig(sshConfig);
sshSession.connect();
ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp");
channel.connect();
return channel;
} catch (JSchException e) {
throw new SftpPoolException("連線sfpt失敗", e);
}
}
@Override
public PooledObject<ChannelSftp> wrap(ChannelSftp channelSftp) {
return new DefaultPooledObject<>(channelSftp);
}
// 銷燬物件
@Override
public void destroyObject(PooledObject<ChannelSftp> p) {
ChannelSftp channelSftp = p.getObject();
channelSftp.disconnect();
}
}
SftpPool.java
package com.vipsoft.sftp.pool;
import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPool;
public class SftpPool<T> extends GenericObjectPool<ChannelSftp> {
public SftpPool(SftpFactory factory) {
super(factory,factory.getProperties().getPool());
}
/**
* 獲取一個sftp連線物件
* @return sftp連線物件
*/
@Override
public ChannelSftp borrowObject() throws Exception {
return super.borrowObject();
}
/**
* 歸還一個sftp連線物件
* @param channelSftp sftp連線物件
*/
@Override
public void returnObject(ChannelSftp channelSftp) {
if (channelSftp!=null) {
super.returnObject(channelSftp);
}
}
}
Utils
ByteUtil.java
package com.vipsoft.sftp.utils;
import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.SftpException;
import com.vipsoft.sftp.exception.SftpPoolException;
import com.vipsoft.sftp.pool.SftpPool;
import java.io.InputStream;
public class SftpUtil {
private SftpPool pool;
public SftpUtil(SftpPool pool) {
this.pool = pool;
}
/**
* 下載檔案
*
* @param dir 遠端目錄
* @param name 遠端檔名
* @return 檔案位元組陣列
*/
public byte[] download(String dir, String name) {
ChannelSftp sftp = null;
try {
sftp = pool.borrowObject();
sftp.cd(dir);
InputStream in = sftp.get(name);
return ByteUtil.inputStreamToByteArray(in);
} catch (Exception e) {
throw new SftpPoolException("sftp下載檔案出錯", e);
} finally {
pool.returnObject(sftp);
}
}
/**
* 上傳檔案
*
* @param dir 遠端目錄
* @param name 遠端檔名
* @param in 輸入流
*/
public void upload(String dir, String name, InputStream in) {
ChannelSftp sftp = null;
try {
sftp = pool.borrowObject();
mkdirs(sftp, dir);
sftp.cd(dir);
sftp.put(in, name);
} catch (Exception e) {
throw new SftpPoolException("sftp上傳檔案出錯", e);
} finally {
pool.returnObject(sftp);
}
}
/**
* 刪除檔案
*
* @param dir 遠端目錄
* @param name 遠端檔名
*/
public void delete(String dir, String name) {
ChannelSftp sftp = null;
try {
sftp = pool.borrowObject();
sftp.cd(dir);
sftp.rm(name);
} catch (Exception e) {
throw new SftpPoolException("sftp刪除檔案出錯", e);
} finally {
pool.returnObject(sftp);
}
}
/**
* 遞迴建立多級目錄
*
* @param dir 多級目錄
*/
private void mkdirs(ChannelSftp sftp, String dir) {
String[] folders = dir.split("/");
try {
sftp.cd("/");
for (String folder : folders) {
if (folder.length() > 0) {
try {
sftp.cd(folder);
} catch (Exception e) {
sftp.mkdir(folder);
sftp.cd(folder);
}
}
}
} catch (SftpException e) {
throw new SftpPoolException("sftp建立目錄出錯", e);
}
}
}
Test
SftpTest.java
package com.vipsoft.sftp;
import com.vipsoft.sftp.utils.SftpUtil;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
@SpringBootTest
public class SftpTest {
@Autowired
private SftpUtil sftpUtil;
@Test
void downloadTest() {
byte[] dockerfiles = sftpUtil.download("/opt/demo/", "Dockerfile");
System.out.println("FileSize =>" + dockerfiles.length);
}
}
