Menu
Woocommerce Menu

我是在调用连接的时候去检查连接是否还存活着,关闭操作主要分为两个步骤

0 Comment


想要模仿zabbix的oracle插件orabix来贯彻对db2的监察,然则Java技术有限,就用python来促成了。不过python常用的连接池PooledDB犹如并不帮忙db2,一贯报那样的不当:”Database
module is not
thread-safe.”所幸作者只是用来做监察和控制的,须求并不是极高,只要达成连接池的三个基本功能就能够:
1.连连复用
2.总是检查,重连

背景

在我们的支出中“池”的定义并不罕有,有数据库连接池、线程池、对象池、常量池等等。上面大家最首要针对线程池来一步一步爆料线程池的面纱。

 

1. 结构体

使用线程池的受益

1、裁减财富消耗

能够另行利用已创设的线程缩小线程创造和销毁形成的开支。

2、进步响应速度

当任务达到时,职分能够不需求等到线程创制就能够马上执行。

3、升高线程的可管理性

线程是稀缺财富,如果无界定地创立,不唯有会损耗系统财富,还有只怕会回降系统的平稳,使用线程池能够开展统一分配、调优和监察

 1 #!/usr/local/bin/python 2 # -*- coding: utf-8 -*- 3  4 import threading,ibm_db,time,Queue 5  6 class db2pool(object): 7   def __init__(self,host,port,user,password,db,conn_num): 8     self.host=host 9     self.port=port10     self.user=user11     self.password=password12     self.db=db13     self.conn_num=conn_num14     15     self.conn_queue=Queue.Queue(0)16             17     for i in range (0,self.conn_num):18       try:19         conn=(ibm_db.connect("DATABASE="+self.db+";HOSTNAME="+self.host+";PORT="+self.port+";PROTOCOL=TCPIP;UID="+self.user+";PWD="+self.password+";", "", ""))20         self.conn_queue.put(conn)21       except Exception as e:  22         print e23   24   def getconnect(self):25       26     for i in range (0,5):27 28       if self.conn_queue.qsize()>0:29         conn=self.conn_queue.get()30         31         try:32           stmt=ibm_db.prepare(conn,"select 1 from sysibm.sysdummy1")33           ibm_db.execute(stmt)34           return conn35         except Exception as e:36           print e37           try:38             conn=(ibm_db.connect("DATABASE="+self.db+";HOSTNAME="+self.host+";PORT="+self.port+";PROTOCOL=TCPIP;UID="+self.user+";PWD="+self.password+";", "", ""))39             self.conn_queue.put(conn)40           except Exception as e:41             print e42         43       time.sleep(0.5)44         45     self.logger.echo("warning","get connection error")46       47   def getclose(self,conn):    48     self.conn_queue.put(conn)49 50 #example51 def work(pool,i):52   # while True:53   conn=pool.getconnect()54   if conn:55     try:56       pass57       stmt=ibm_db.prepare(conn,"select "+str(i)+" from sysibm.sysdummy1")58       ibm_db.execute(stmt)59       result=ibm_db.fetch_both(stmt)60       while (result):61         print "\nresult[0]="+str(result[0])+"++++"+str(conn)62         result=ibm_db.fetch_both(stmt)63     except Exception as e:64       print e65     finally:66       pass67       pool.getclose(conn)68   time.sleep(5)69 70 if __name__=="__main__":71   pool=db2pool("172.16.2.9","60000","db2inst1","db2inst1","secs",5)72   for i in range (0,50):73     t=threading.Thread(target=work,args=(pool,i,))74     t.start()

1.1 depSet和finalCloser

depSet : 记录db与conn之间的依附关系,维持连接池以致关闭时利用
finalCloser: Todo

 // depSet is a finalCloser's outstanding dependencies
 type depSet map[interface{}]bool // set of true bools
// The finalCloser interface is used by (*DB).addDep and related
// dependency reference counting.

type finalCloser interface {
// finalClose is called when the reference count of an object
// goes to zero. (*DB).mu is not held while calling it.
finalClose() error

1.2 DB结构体

// DB is a database handle representing a pool of zero or more
// underlying connections. It's safe for concurrent use by multiple
// goroutines.
//
// The sql package creates and frees connections automatically; it
// also maintains a free pool of idle connections. If the database has
// a concept of per-connection state, such state can only be reliably
// observed within a transaction. Once DB.Begin is called, the
// returned Tx is bound to a single connection. Once Commit or
// Rollback is called on the transaction, that transaction's
// connection is returned to DB's idle connection pool. The pool size
// can be controlled with SetMaxIdleConns.
type DB struct {
    driver driver.Driver
    dsn    string
    // numClosed is an atomic counter which represents a total number of
    // closed connections. Stmt.openStmt checks it before cleaning closed
    // connections in Stmt.css.
    numClosed uint64
    mu           sync.Mutex // protects following fields
    //当前连接池中空余的连接
   //当freeConn>maxIdle时,会将多余的那部分连接close掉
    freeConn     []*driverConn
    connRequests []chan connRequest
    numOpen      int // number of opened and pending open connections
    // Used to signal the need for new connections
    // a goroutine running connectionOpener() reads on this chan and
    // maybeOpenNewConnections sends on the chan (one send per needed connection)
    // It is closed during db.Close(). The close tells the connectionOpener
    // goroutine to exit.
    openerCh    chan struct{}
    closed      bool
    // 记录依赖关系
    dep         map[finalCloser]depSet
    lastPut     map[*driverConn]string // stacktrace of last conn's put; debug only
    //maxIdle和maxOpen的关系?
   // maxIdle为连接池中最大的连接数目
   // maxOpen为打开db最大的连接数 : maxOpen仅当>0且< maxIdle的情况下才会生效
    maxIdle     int                    // zero means defaultMaxIdleConns; negative means 0
    maxOpen     int                    // <= 0 means unlimited
    //单个连接最大生命周期
    maxLifetime time.Duration          // maximum amount of time a connection may be reused
    cleanerCh   chan struct{}
}

// connReuseStrategy determines how (*DB).conn returns database connections.
type connReuseStrategy uint8

const (
    // alwaysNewConn forces a new connection to the database.
    alwaysNewConn connReuseStrategy = iota
    // cachedOrNewConn returns a cached connection, if available, else waits
    // for one to become available (if MaxOpenConns has been reached) or
    // creates a new database connection.
    cachedOrNewConn
)

1.3 driverConn结构体
driverConn为单个连接的构造体,同期它有一个其依赖的db的指针(因为每一种连接都是创造在db之上),在DB中能够拍卖单个连接的操作(举个例子Close(卡塔尔国操作)

// driverConn wraps a driver.Conn with a mutex, to
// be held during all calls into the Conn. (including any calls onto
// interfaces returned via that Conn, such as calls on Tx, Stmt,
// Result, Rows)
type driverConn struct {
    db        *DB
    createdAt time.Time

    sync.Mutex  // guards following
    ci          driver.Conn
    closed      bool
    finalClosed bool // ci.Close has been called
    openStmt    map[driver.Stmt]bool

    // guarded by db.mu
    inUse      bool
    onPut      []func() // code (with db.mu held) run when conn is next returned
    dbmuClosed bool     // same as closed, but guarded by db.mu, for removeClosedStmtLocked
}

线程池的工作规律

率先咱们看下当二个新的天职交给到线程池之后,线程池是什么样管理的

1、线程池判定大旨线程池里的线程是或不是都在实行任务。假设不是,则创建贰个新的干活线程来试行职分。假如宗旨线程池里的线程都在施行职分,则施行第二步。

2、线程池判定工作行列是不是曾经满。假设工作行列没有满,则将新交付的职务存款和储蓄在这里个职业行列里开展等待。若是职业行列满了,则执行第三步

3、线程池剖断线程池的线程是或不是都远在专业情景。若无,则开创二个新的工作线程来实行义务。假若已经满了,则交给饱和计谋来拍卖那些职责

 

2. 连接池原理深入分析

线程池饱和布署

此地提到了线程池的饱和战略,那大家就简介下有哪些饱和攻略:

AbortPolicy

为Java线程池默许的鸿沟计策,不奉行此任务,而且一贯抛出二个运作时极度,切记ThreadPoolExecutor.execute须求try
catch,否则程序会直接退出。

DiscardPolicy

平昔放弃,任务不奉行,空方法

DiscardOldestPolicy

从队列之中舍弃head的八个职责,并再度execute 此task。

CallerRunsPolicy

在调用execute的线程里面实施此command,会堵塞入口

顾客自定义拒绝战略(最常用)

完毕RejectedExecutionHandler,并自身定义计策方式

下大家以ThreadPoolExecutor为例展现下线程池的工作流程图

图片 1

图片 2

1、假使当前运营的线程少于corePoolSize,则创制新线程来进行职务(注意,实施这一步骤需求获得全局锁)。

2、若是运转的线程等于或多于corePoolSize,则将任务加入BlockingQueue。

3、假若不可能将职责参与BlockingQueue(队列已满),则在非corePool中开立异的线程来拍卖职责(注意,试行这一步骤需求拿到全局锁)。

4、假若创立新线程将使近年来运作的线程超过maximumPoolSize,职责将被反驳回绝,并调用RejectedExecutionHandler.rejectedExecution(卡塔尔(قطر‎方法。

ThreadPoolExecutor选用上述手续的总体规划设计思路,是为了在试行execute(卡塔尔(قطر‎方法时,尽也许地防止获取全局锁(这将会是三个严重的可伸缩瓶颈)。在ThreadPoolExecutor完毕预热之后(当前运作的线程数大于等于corePoolSize),大致具备的execute(State of Qatar方法调用都以推行步骤2,而步骤2无需得到全局锁。

发表评论

电子邮件地址不会被公开。 必填项已用*标注

相关文章

网站地图xml地图