Mass testing and characterization of 20-inch PMTs for JUNO

Abstract

Main goal of the JUNO experiment is to determine the neutrino mass ordering using a 20 kt liquid-scintillator detector. Its key feature is an excellent energy resolution of at least 3% at 1 MeV, for which its instruments need to meet a certain quality and thus have to be fully characterized. More than 20,000 20-inch PMTs have been received and assessed by JUNO after a detailed testing program which began in 2017 and elapsed for about four years. Based on this mass characterization and a set of specific requirements, a good quality of all accepted PMTs could be ascertained. This paper presents the performed testing procedure with the designed testing systems as well as the statistical characteristics of all 20-inch PMTs intended to be used in the JUNO experiment, covering more than fifteen performance parameters including the photocathode uniformity. This constitutes the largest sample of 20-inch PMTs ever produced and studied in detail to date, i.e. 15,000 of the newly developed 20-inch MCP-PMTs from Northern Night Vision Technology Co. (NNVT) and 5000 of dynode PMTs from Hamamatsu Photonics K. K.(HPK).

1 Introduction

The Jiangmen Underground Neutrino Observatory (JUNO) experiment [1, 2] is a new large-volume multi-purpose liquid-scintillator experiment currently under construction in southern China, located in a cavern with a 700 m rock overburden. Its main goal is to determine the neutrino mass ordering from the neutrino oscillation spectrum of two close-by nuclear power plants at a distance of 53 km each, with a sensitivity better than three standard deviations after six years of data taking [2,3,4]. The central detector (CD) of JUNO [5,6,7,8] (Fig. 1) consists of an acrylic sphere with a diameter of 35.4 m and is filled with 20 kt of LAB-based liquid scintillator (LS) [9]. A high transparency LS, high optical coverage (>75%) with high quantum efficiency photomultiplier tubes (PMTs), and low background/noise levels are needed to achieve an energy resolution of at least 3% at 1 MeV [8, 10], which is essential for realizing the physics goals of JUNO. The high coverage of the CD is achieved by closely packing 17,612 high quantum efficiency 20-inch PMTs (Large PMTs or LPMTs) and 25,600 3-inch PMTs (Small PMTs or SPMTs) in the gaps between the LPMTs, placed at a distance of 19.8 m to the detector’s center. All of them are mounted on a stainless steel lattice shell (diameter of 40.1 m) outside the acrylic sphere. The whole construction is further embedded into a cylindrical pure water pool with a diameter of 43.5 m and depth of 44 m (water filled to 43.5 m), instrumented with another about 2400 20-inch PMTs. The water pool will be used as an active Cherenkov veto against cosmic muons traversing the detector, but also acts as a shielding buffer from the surroundings.

Fig. 1

Layout of the JUNO detector system [8]. The zoomed in picture on the right is illustrating the proposed PMT arrangement of the JUNO central detector, with the 3-inch PMTs placed in the gaps between the 20-inch PMTs

In large volume LS detectors as JUNO, optimizing the energy resolution specifically concerns all parts related to the light collection and detection, including in particular the performance of the light-sensitive devices in the range from single photo-electrons (SPE) to hundreds of photo-electrons per channel. Aiming to achieve an unprecedented energy resolution, JUNO is designed with a set of 1 GHz waveform sampling electronics to record the pulses of the 20-inch PMTs for a precise neutrino event detection and reconstruction [8, 11, 12]. Playing the key role in this process, the PMTs’ photon detection efficiency, dark count rate (DCR), and timing characteristics are most important to achieve the best possible detector performance [8]. At the same time, good performance in other parameters is needed for better event reconstruction, background reduction, and a successful long-term operation: this concerns the gain as a function of high voltage, charge resolution, peak-to-valley ratio, pulse shape features, correlated pulses (pre-/after-pulses), and, in particular, the photocathode response uniformity and Earth’s magnetic field (EMF) sensitivity of the large volume PMTs. Detailed knowledge of the PMT characteristics moreover provides valuable input for precise detector simulations leading to a better understanding of the to-be-running experiment [8, 13,14,15,16]. All LPMTs used in JUNO have been measured in detail and checked against a list of preassigned requirements (see Sect. 2.2). These tests also include consistency checks between bare and waterproof potted PMTs.¹

Concerning the mechanical safety which is relevant for the large vacuum glass bulbs of PMTs working in pure water at a maximum depth of 44 m, a careful inspection is performed on each PMT to identify any mechanical/structural defects. Furthermore, following many valuable, previously performed studies on different PMT parameters, such as general studies on the large area PMT performance [19,20,21,22,23,24,25], temperature effects [26,27,28,29], and the lifetime of the newly developed MCP-PMT by Northern Night Vision Technology Co. (NNVT) [30,31,32,33], follow-up studies on the DCR as a function of temperature and the long-term stability of the MCP-PMTs are evaluated.

Fig. 2

Technical drawing of selected PMTs for JUNO: NNVT 20-inch MCP-PMT (type GDB-6201 [63], left) and HPK 20-inch dynode PMT (type R12860 [59], right)

In this work, all details are presented about the measured characteristics of all accepted JUNO 20-inch PMTs, as well as brief introductions to the setup of the mass testing systems and the performed testing procedures. The selected 20-inch PMT types of JUNO will be described in Sect. 2. The testing facilities and performed procedures will be presented in Sect. 3. The testing results of the bare PMTs (full 20,000 LPMT sample) will be shown in detail in Sect. 4. Expected features of waterproof potted PMTs and differences with respect to testing results from the acceptance tests of the bare PMTs will be discussed in Sect. 5. Finally, a summary is given in Sect. 6. Meanwhile, it is valuable to stress that several other PMT characterization campaigns were performed in the past; we mention those of the IceCube [34], MiniBooNE [35], IMB [36], Borexino [37, 38], Daya Bay [39], Chooz [40], SNO [41], Double Chooz [42, 43], DAMA [44], Auger [45, 46], ANNIE [47], YBJ [48], LHAASO [49, 50], Super-K [51], Hyper-Kamiokande [52, 53], and KM3Net [54, 55] experiments. For completeness, it should be noted here that the 3-inch PMTs used for JUNO are investigated as well in a separate campaign [56, 57].

2 20-inch photomultiplier tubes for JUNO

2.1 Selected 20-inch PMTs of JUNO

It is theoretically and practically significant to build a detector with maximum physics potential at minimal costs. To date, PMTs offer the best compromise between single-photon sensitivity and acceptance in noise and cost per unit area around room temperature, and large area PMTs are the first choice of photon sensors for large LS- or water-based neutrino experiments which require high photocathode coverage. Besides photon detection efficiency (PDE), other characteristics such as dark count rate (DCR), transit time spread (TTS), the radioactive background of the glass, peak-to-valley ratio (P/V), etc., will affect the photon detection and event reconstruction, and thus impact the physics measurements. Considering all related PMT parameters, costs and risks, evaluated with the physics goals of JUNO, a selection strategy for PMTs was proposed and applied, leading to a set of requirements for the performance of JUNO 20-inch PMTs [58]. With the bidding following the selection strategy, there are two types of 20-inch PMTs selected by JUNO: approx. 5000 box and linear-focused dynode-PMTs (R12860-50 HQE or R12860) from Hamamatsu Photonics K. K. (HPK) [59] and approx. 15,000 MCP-PMTs (GDB-6201 or N6201) from North Night Vision Technology Co. (NNVT) [60,61,62,63]. The dimensions of the two types of PMTs are shown in Fig. 2.

2.2 Acceptance criteria

Based on joint considerations of the physics requirements discussed above and the PMT manufacturers, a list of combined performance criteria was defined for all ordered 20,000 20-inch PMTs of JUNO. Each of the received PMTs must be tested at least once to assure that all predefined requirements are satisfied. In Table 1 the required nominal values as well as lower or upper limits for part of the parameters are summarized. Each of the listed parameters will be discussed separately and in detail within this paper. The photon detection efficiency (PDE), as one critical parameter required by JUNO and discussed in detail in Sect. 4.4, is not listed directly in Table 1, but appears as a combined parameter of the quantum efficiency (QE), the collection efficiency (CE), and the effective area ratio (EAR).² It is finally requested with an averaged value of $\ge 27\%$ for all the PMTs, and a minimum value of $\ge 24\%$ for a single PMT (in particular for use in the CD), both defined at a wavelength of 420 nm to match the typical emission spectra of JUNO liquid scintillator [9]. A catalog index of the content and all the checked parameters is listed in Table 2.

Table 1 Main acceptance criteria for JUNO 20-inch PMTs

Table 2 Catalog index of content and checked parameters

3 Testing setup and procedure

3.1 Testing facilities

To concentrate all actions related to PMT testing, a warehouse with around 4500 $\text {m}^2$ was rented since 2017 at Zhongshan Pan-Asia Electric Co., Ltd., Guangdong Province, China, which is about 150 km away from the JUNO experimental site. In this warehouse, named Zhongshan Pan-Asia 20-inch PMT testing and potting station, all relevant steps of preparing the 20,000 20-inch PMTs for JUNO were processed, including receiving (database management for recording and statistics), acceptance tests, storage, and waterproof potting. The warehouse runs in a controlled environment with a temperature of $25^{\circ }$C (in a range of $\pm 3^{\circ }$C) and relative humidity of 50% (in a range of 30–70%), and is maintained by a local team which is also in charge of the daily PMT testing routine and all storage operations. At the same time, most of the 20-inch PMT testing shifts focusing on data acquisition and analysis are shared by the whole JUNO collaboration.

Based on the specific criteria of the selected 20-inch PMTs, a set of acceptance testing facilities were developed to realize a process of semi-automatic PMT testing covering most of the parameters listed in Table 1.³ Main units are a commercial-container-based multi-channel testing system developed and installed to perform full characterizations for each of the 20,000 20-inch PMTs, as well as a scanning station system designed and built aiming to sample about $5\%$ of all the PMTs to check i.e. their photocathode uniformity. All PMT tests are performed under a shielding against Earth’s magnetic field (EMF) with a remaining strength of less than $10\%$ of EMF (which is about 5 µT).

All PMTs are tested with JUNO-optimized high voltage (HV) dividers using a positive HV [18, 65,66,67]. The initial tests of the full sample of PMTs used a pluggable HV divider with an integrated HV-signal de-coupler on board. Later, a large sub-sample of the PMTs were tested again with the final JUNO version of the HV divider directly soldered to the PMT pins, where the entire PMT and HV divider were encapsulated with a waterproof housing [18, 68]. In the latter case the PMT pulses are picked up from a stand-alone HV-signal decoupler through a 2 m (central detector PMTs) or 4 m (water pool PMTs) cable.⁴ Although these versions differ in their mechanical setup, they feature the same HV-divider ratio and direct current (DC). However, the first few thousand bare NNVT PMTs were measured by another HV-divider⁵ which only affects the rise- and fall-time of the pulses of these PMTs.

Two containers and two scanning stations are built and used for the bare PMT testing. Another two containers of slightly modified interior design are set up, one dedicated to long-term stability tests of PMTs, and another to the testing of waterproof potted PMTs together with JUNO (final front-end) 1F3 electronics [8].

3.1.1 PMT mass testing: container systems

The main setup of the individual (acceptance) tests of all 20,000 20-inch JUNO PMTs is the so-called container system. This system has been described extensively in a stand-alone paper [69], where all details about mechanical setup, data taking electronics, measurement process and accuracies can be found. To enhance understanding of the later presented results from the PMT mass testing, the setup will be briefly described here as well.

The container system consists of four 20-feet high-cube reefer containers, which act as darkrooms and are able to host 36 (containers $\#$A and $\#$B) or 32 (containers $\#$C and $\#$D) PMTs in optically separated measurement channels (shelf system with drawer boxes), see Fig. 3. Each container is passively shielded against the EMF by a multi-layer silicon-iron shielding and equipped with a high power HVAC (heating, ventilation, and air conditioning) unit, granting environmental control over the container’s interior. All drawer boxes are further equipped with temperature sensors, which allow to measure the relationship between dark count rate and temperature of the surroundings. Containers $\#$A and $\#$B are equipped with commercial data taking electronics (main unit is a CAEN V1742 switched-capacitor digitizer [70]), controlled and supervised by a custom-made data acquisition software (DAQ) based on LabView. This DAQ performs a sequence of individual measurements to enable a full PMT characterization within 24 h, covering absolute PDE, DCR, gain, TTS,⁶ P/V, charge resolution, pulse characteristics such as rise-/fall-time, pre-pulse ratio and signal-to-noise ratio. These containers are intended to perform the acceptance tests of the full LPMT sample as well as functionality tests of a large sub-sample of potted PMTs. Container $\#$C is equipped with a similar set of commercial electronics and intended to act as long-term stability test setup, while container $\#$D is modified to host the JUNO 1F3 underwater boxes [8], allowing additional functionality and performance tests of waterproof potted PMTs together with the final JUNO electronics.

Each drawer in all four containers is equipped with a LED device from company HVSys [71], together with a collimator, attenuator, diffuser, and a cylindrical reflector to illuminate the full photocathode uniformly for SPE- and multi-p.e.-related measurements at optimized light intensities [72] and uniformity [69] as shown at the bottom of Fig. 3. These devices are pulsed LEDs (at 420 nm) featuring a feedback loop on a small micro-controller, which stabilizes the light intensity over time to a calibrated level (based on a dedicated calibration campaign prior to the testing campaign, see Sects. 4.2.1 and 4.4 for details). Each of the containers $\#$A and $\#$B are further equipped with a picosecond laser flashing system based on a PiLas 420X picosecond laser from Advanced Laser Systems (A.L.S.) [73]. This device can produce short light pulses with a wavelength of 420 nm and a width of $\sim 80$ ps, which are distributed by a fiber splitter over all channels and used for precise TTS measurements at SPE level.⁷ In container $\#$C specifically, an additional set of LEDs was introduced into the drawer boxes pulsing with different frequencies and light intensities in order to simulate an accelerated aging of the PMTs due to higher stress.

The performance and stability of each container (except for container $\#$C) is monitored by a small number of 20-inch PMTs (“reference PMTs” or “monitoring PMTs”, consisting of up to three HPK and two NNVT PMTs per container), which are included in every measurement run with one staying at a fixed drawer and the others being circulated over all channels. The containers achieve a stable performance over time as well as a sufficient accuracy of e.g. $\le 1\%$ for the absolute PDE and $\le 1$ ns for the timing parameters such as the TTS (only container $\#$B) [69] – this will be discussed also in Sects. 4.4 and 4.6. Hence, the system can provide reliable and comparable results for all available 20-inch PMTs characterized in this PMT testing campaign. More details about this system can be found also in [69, 74, 75].

Fig. 3

Pictures of the PMT container system, located at Zhongshan Pan-Asia 20-inch PMT testing and potting station, China. Top row: Conceptual and actual picture of the four testing containers; the sketch drawing shows the shelf system mounted inside. Bottom left: Interior of one of the testing containers, with a PMT loaded onto a drawer. Bottom right: Schematic view of the inside of a drawer box

3.1.2 PMT sample testing: scanning stations

The second setup for the individual testing of a large sub-sample of 20-inch JUNO PMTs is the so-called scanning station system. Also this system has been described already in other publications [75, 76], where details about mechanical setup, electronics, and measurement process can be found. Similar to before, this setup shall be briefly described, to enhance understanding of the later presented results.

The scanning station system constitutes a complementary system, which is able to investigate possible inhomogeneities of characteristics along the PMT’s photocathode surface (in complementary to the container system, which is not sensitive to local inhomogeneities). This can be achieved using a rotatable arch with seven stabilized pulsed LEDs mounted at different zenith angles (#1 (pole), $\ldots$, #7 (equator)) covering all 360$^{\circ }$ azimuthal angles in 15$^{\circ }$ steps and thus the full photocathode surface. The light sources used by the scanning stations are LEDs also from HVSys company (similar to the ones in the container systems) but only equipped with collimators and attenuators (detected light intensity is 1 $\sim 1.5$ p.e. per flash). Two of these scanning stations are set up for mass sampling and operated in separate darkrooms (Fig. 4). Due to an active magnetic compensation of the scanning stations provided by Helmholtz coils installed within the walls, floor, and ceiling of the housing darkrooms of each station, they also allow for the testing of a PMT’s magnetic field sensitivity in a range from $-50$ µT to $+50$ µT as well as a PMT characterization in a completely EMF-free environment.⁸ The scanning station uses a DRS4-based ADC [77] with an additional $\times$ 10 amplifier to acquire full PMT waveforms. The scanning procedure and data analysis are fully automated in a four to six hours cycle per PMT and station. The absolute light intensities of all LEDs in both scanning stations are frequently calibrated and monitored using a small calibration PMT (head-on-type linear-focused $1\frac{1}{8}$-inch R1355 PMT with a PDE of $23.9\pm 1.0\%$ at 420 nm). The calibrated LED intensity only shows a spread of less than 2% relatively over the whole running period and thus proves the high stability in light output of the LEDs over time.

The scanning stations allow the characterization of individual PMTs in all relevant parameters by scanning the PMT’s photocathode (i.e. local gain and average number of detected photo-electrons), and enable a deeper understanding of their performance based on the PMT’s uniformity (see also Sect. 4.10). This may help to further understand the PMTs’ performance and identify potential problems not detectable by only using the containers as testing facilities. The scanning stations are moreover used to define the weighted photon detection efficiency (PDE) of the whole 20-inch photocathode surface, which is one of the key parameters to be discussed in Sect. 4.4. Since the characterization process is more complex, and is complementary to the container system each scanning station can take only one PMT to be tested, these features will be obtained only for a sub-sample $\sim 5\%$ of all 20,000 PMTs (originally aimed to cover about 5% randomly selected, added by cross-checks between containers and scanning stations).

Fig. 4

Pictures of the scanning station system. Top: the darkrooms used for the two scanning stations; bottom: mechanical setup and rotating support of a single scanning station

3.2 Receiving, cataloging and testing procedures

Starting from early 2017, the first batches of JUNO 20-inch PMTs arrived at Zhongshan Pan-Asia 20-inch PMT testing and potting station, and continued to do so over a period of about four years (usually in batches of several hundred PMTs) (see Fig. 5). After their arrival, a checklist was applied step by step to realize a detailed characterization and full documentation of all PMTs. This includes the whole process of checking-in, multiple testings (including visual inspections as well as performance tests with the containers and scanning stations), waterproof potting, and storage until further processing. Barcodes representing the PMT ID are used as a keyword to identify all individual records. The management of all PMTs is handled using a database with a web page link [78] following the scheme in Fig. 6, which includes the vendor data of each PMT, inspection results, measurement records of both container and scanning station systems, measurement results management, waterproof potting status, potted PMT testing results, and storage location in individual charts, everything tagged by the individual unique PMT ID. Generally, it takes about three to six months to complete the whole process following the PMT receiving and storage. This cautiously executed procedure then leads to a final classification (accepted to use in JUNO or rejected). Since PDE and DCR are critical parameters, they will be cross-checked between containers and scanning stations if the initially measured value is around the boundary of the requirements as listed in Table 1 and indicated in Fig. 6).

Fig. 5

Statistics of PMTs received at Zhongshan Pan-Asia 20-inch PMT testing and potting station and later tested within the testing systems. In total 22,414 PMTs were delivered

Fig. 6

Measurement and classification procedure performed by the two testing systems. Details of the full PMT characterization process performed by the container system can be also found in [69]

During the whole PMT testing campaign, 22,414 PMTs have been received and stored in the warehouse, more than 52,000 individual PMT characterizations with the containers and more than 5200 PMT scans with the scanning stations have been completed. Some PMTs have been tested multiple times including the systems’ calibrations, bare PMT testing, tests of the monitoring PMTs, potted PMT testing (using commercial electronics and JUNO 1F3 electronics) and special testing campaigns (see Fig. 7). In total, 20,065 PMTs have finally passed all testing criteria and then are selected to operate in the JUNO experiment.

Fig. 7

Time evolution of the PMT testing. Left: more than 52,000 individual PMT characterizations were performed in about 1600 runs using containers $\#$A, $\#$B and $\#$D (some PMTs have been tested multiple times including calibration runs, acceptance tests, re-tests, and continuous measurement campaigns). Right: 4656 valid characterizations from more than 5200 scans (including special testings, and calibration) were performed using the scanning stations $\#$1 and $\#$2

4 PMT testing and characterization: bare PMT results

With the designed testing systems, each of the received PMTs has been tested at least once for the required parameters. Their performance will be discussed in detail in the following sections, leading to the selection of a sample of individually qualified PMTs for JUNO. During the daily testing, the reference PMTs (compare Sect. 3.1.1) are loaded into the containers in each run together with the untested PMTs to monitor the comparability of the measured parameters (and i.e. the stability of the container performance) over time.

4.1 Visual inspection

When a 20-inch PMT is working in up to 44 m water depth, there is an implosion risk for its evacuated glass bulb [79,80,81]. To protect the PMTs from a possible implosion chain triggered by any accident or defective PMT, all PMTs will be capsuled using a protection cover design featuring an acrylic shell on its upper half and a stainless steel shell around its bottom half [8, 82]. The protection cover was optimized considering its structural strength, a good transparency, dimension limitations from PMT photocathode coverage and installation clearance, a compatibility with pure water, and a low radioactivity [82]. The smaller diameter of PMT will affect the physics coverage and the fix structure of the installation, while the larger diameter is possible to introduce some conflicts among PMTs. In order to satisfy the needs for the protection covers and assure an installation tolerance on the $\sim 3$ mm level, a precise knowledge of the dimensions of all individual PMTs is required. Therefore, the dimensions (diameters) of the PMTs’ glass bulbs were measured for all individual PMTs. The typical diameter of HPK dynode PMTs was found to be 510 mm in a range of 505–512 mm; the typical diameter of NNVT MCP-PMTs was found to be 509 mm in a range of 506–511 mm (Fig. 8). Only four HPK PMTs are out of the tolerance range, which has negligible impact on both physics and installation.

Fig. 8

Measured diameter in mm of all checked 20-inch PMTs. Some PMTs were rejected before the measurement after visual inspection, therefore the total number of tested PMTs is slightly reduced. Left: HPK; Right: NNVT

The glass thickness⁹ is assumed to be proportional to the PMT weight statistically, thus measuring the PMTs’ weight helps to evaluate possible weaknesses, where the lighter PMTs can be cross-checked further in more detail.¹⁰ According to the measurement on a sub-sample of all PMTs, the typical weight of an HPK dynode PMT is about 7.4 kg in a range of 6.5–8.4 kg; the typical weight of an NNVT MCP-PMT is about 7.5 kg in a range of 5.2–9.1 kg (see Fig. 9).

Fig. 9

Measured weight in kg of all checked 20-inch PMTs. Weighing was performed only with a sub-sample of all PMTs. Left: HPK; right: NNVT

A high-quality glass bulb is essential to avoid any dangerous defect related to its structural strength, especially for JUNO since all PMTs are closely packed. Even though both vendors have improved their glass bulb’s strength via an optimized geometry as well as the identification of any small bubbles, cracks, weaknesses on the glass thickness, etc., a detailed visual check on every PMT is still required. Thus, all individual PMTs were visually checked for mechanical related issues (including cracks, bumps, scratches), bubbles and open bubbles, broken holes (glass notches) on the outside glass surface, defects at the sealing or KOVAR section¹¹ (including a gap between the KOVAR plate and the glass, dislocation of the KOVAR plate, broken glass or glass shedding at the transitional section of NNVT PMTs, and issues related to the socket), glass devitrification, gas leakage, and all other possible defects. Furthermore, internal impurities such as metal contaminants in the photocathode or glass were checked since they might affect the electronic features of the PMTs. The results of the visual inspection are listed in Table 3 including several PMT defects.

Table 3 Main defects of all 20-inch PMTs rejected after the visual inspection

The visual inspection is a primary step of the PMTs’ acceptance testing procedure. All PMTs with good tags are transferred to the following performance checks. In total, 5041 PMTs of 5414 delivered HPK PMTs, and 16,357 PMTs from 17,000 delivered NNVT PMTs were selected as they satisfied the requirements of the visual inspection for JUNO.

4.2 Working high voltage and gain

4.2.1 Charge spectrum calculation

Using the containers, all PMT waveforms are acquired by a 1 GS/s (samples per second) Analog-Digital Converter (ADC, type CAEN V1742) in 512 ns long window, synchronized to the auto-stabilized low intensity LED light pulses. To calculate the charges of the PMT pulses, each waveform is integrated in a time window $[-20,+55]$ ns relative to the peak location of the signal (see Fig. 10a), considering the external trigger timing, an input impedance of 50 $\Omega$ of the electronics [83]. An example waveform and charge spectrum in pC is shown in Fig. 10. The single photo-electron (SPE) spectrum can be de-convoluted with Poisson statistics as in Eq. 1, where µ is the mean p.e. number received by the first dynode or MCP, and P(n) is the probability to have n p.e.s. When the value of µ is around 0.1, the probability P(1) of 1 p.e. events is 0.090 (and $P(2) \sim 0.005$ for 2 p.e. events etc.), thus single p.e. events will dominate the signal distribution in this case. The resulting distribution or SPE response spectrum (SPR) can be used in the following analyses of e.g. the gain (using a comparable and constant light intensity of µ $\sim 0.1$ for all LEDs) or the PDE (using individual light intensities and a calibration function for a set DAC of each LED, which is calibrated using a small sample of 20-inch reference PMTs¹² with known PDE):

$$\begin{aligned} P(n)=\frac{e^{-\mu }\mu ^n}{n!}. \end{aligned}$$

(1)

In the resulting charge spectrum (see Fig. 10b), the pedestal represents all trigger/signal events with no light; it will be fitted in the range $[-0.15,+0.15]$ p.e. ($Q_{\textit{ped}}$ and $\sigma _{\textit{ped}}$). The second peak right of the pedestal peak corresponds to single p.e. events, located around 1.602 pC for a gain of $1\times 10^{7}$ (as assumed in this example). This SPE peak will be fitted in the range [0.5, 1.4] p.e.¹³ ($Q_{sig}$ and $\sigma _{sig}$). All the fitting mentioned here is done with a Gaussian function.

Fig. 10

Typical plots of an averaged waveform from a single PMT (left, also showing the typical definitions used for the following analysis) and the corresponding integrated charge spectrum from these waveforms (right). The SPE peak is located around 1.602 pC in the case of a gain of $1\times 10^7$

4.2.2 Gain and HV determination

To calculate the PMT gain G at a set voltage, the information from the charge spectrum as shown before in Fig. 10b can be used as shown in Eq. 2:

$$\begin{aligned} G = \frac{Q_{\textit{sig}}-Q_{\textit{ped}}}{e}. \end{aligned}$$

(2)

The voltage to be applied for a gain of $1 \times 10^7$ is determined from a series of measurements with different high voltages (HV), performed between $-150$ V to $+150$ V (executed in 50 V steps) relative to the HV value proposed by the manufacturer. The optimal working HV for a $1\times 10^{7}$ gain is then identified using a fit between applied HV and measured gain, as shown in Fig. 11a. The used fitting function is defined in Eq. 3 with G the measured gain, which is consistent with the applied fitting function used for dynode PMTs in the selected range [28]. Additional information about the HV determination can be found in [69]. A discussion about different methods for the gain determination and their effects can be found also in separate papers [24, 84, 85]:

$$\begin{aligned} \log _{10}\,(G) = A\times {\textit{HV}}+B. \end{aligned}$$

(3)

The fitted correlation factor A between HV and gain G of all the measured PMTs is shown in Fig. 11b. The mean value of A is 0.0013 for HPK and 0.0022 for NNVT, which means the working gain will be doubled when increasing the HV by additional $\sim$ 230 V for HPK PMTs and $\sim$ 140 V for NNVT PMTs around this working gain range.

Fig. 11

Example plot for a HV vs. gain fit (left, for a single PMT) and the distribution of the fitted gain factor A (as introduced in Eq. 3) of all qualified PMTs (right)

With having tested the monitoring PMTs hundreds of times during the whole campaign, the standard deviation of the determined working HV repeatability was found to be about 1% $(15\sim 18$ V) for both HPK and NNVT PMTs on average, and the standard deviation of the determined $1\times 10^7$ gain is 2% for HPK dynode PMTs and 3% for NNVT MCP-PMTs, which indicates the uncertainty in determining the working HV and gain with the container system.

The determined working HV of all qualified bare PMTs producing a gain of $1 \times 10^7$ is shown in Fig. 12a; the determined mean HV is about 1863 V for HPK, and 1748 V for NNVT PMTs. The determined HV for a $1\times 10^{7}$ gain is a slightly biased with respect to the values suggested by the manufacturers of both PMT types (the working HV is 20 V higher for HPK PMTs and 20 V lower for NNVT PMTs on average compared to the supply voltages suggested by the manufacturers), even though they show a very good correlation (the fitted slope is 0.99 for HPK PMTs and 1.01 for NNVT PMTs), as illustrated in Fig. 13. This difference could arise from different testing methods, cable attenuations, used HV dividers and electronics. However, all PMTs are reaching the requirements for the working HV at a $1\times 10^{7}$ gain, i.e. the NNVT PMTs show a much lower working HV value than the allowed maximum value as fixed in Table 1.

The final gain applied in the container system during the PMT characterization is cross-checked once more as a part of the regular analysis. The results show that the set gains do not exactly match a value of $1\times 10^{7}$ in all cases, but follow a distribution with a width of 2–4% (which is consistent to the estimation about the gain determination uncertainty from the monitoring PMTs) as well as an offset of $3\%$ for the NNVT MCP-PMTs towards a higher gain, as presented in Fig. 12b. This is mainly introduced by the used HV-gain model in the data analysis, which is more applicable for the dynode PMTs than for the MCP-PMTs.

Fig. 12

Determined HV and gain of all qualified PMTs. Measured gains not contained in $[0.95,1.15] \times 10^7$ lead to a retest of the corresponding PMT. Black: all PMTs; red: NNVT; blue: HPK

Fig. 13

Comparison between the working HV for a gain of $1\times 10^{7}$ of all qualified bare PMTs, measured by JUNO and the suggested HV values from the manufacturers. Left: HPK; right: NNVT

4.3 Single photo-electron features

With the determined HV for a $1\times 10^7$ gain, approximately 20,000 waveforms are acquired for each PMT in the SPE mode (with µ$~\simeq 0.1$ p.e.) using an external trigger synchronizing the LED pulses to the data acquisition. The recorded waveforms are analyzed for their characteristic features in order to extract the SPE spectrum and a number of important parameters characterizing the PMT’s performance. These parameters and the resulting distribution for the full data set will be presented in following sections.

4.3.1 Signal-to-noise ratio (S/N)

The signal-to-noise (S/N) ratio in the container measurements is defined as a control parameter to ensure a valid measurement of the charge resolution and peak-to-valley ratio, as well as for a reliable threshold setting in the following PDE and DCR measurements. The S/N ratio is defined in Eq. 4, where $\sigma _{\textit{ped}}$, $Q_{\textit{sig}}$ and $Q_{\textit{ped}}$ all are calculated as described in Sect. 4.2.1. The S/N ratio is required to be larger than 10 in all individual SPE measurements, corresponding to a noise level smaller than 0.1 p.e. At the container systems, the mean S/N ratio for both HPK and NNVT PMTs is around 13, as depicted in Fig. 14a.

For comparison, the scanning station has reached a S/N ratio of 20 in its measurements due to the use of an additional amplifier (see Sect. 3.1.2). This also allows high quality cross-checks of the PDE and DCR measurement with the scanning station:

$$\begin{aligned} S/N=\frac{Q_{\textit{sig}}-Q_{\textit{ped}}}{\sigma _{\textit{ped}}}. \end{aligned}$$

(4)

Fig. 14

The S/N ratio of all the measurements with qualified PMTs (left). The SPE amplitude of all qualified PMTs (right). Black: all PMTs; red: NNVT; blue: HPK. Note: the amplitude is only selected within [4, 10] mV

4.3.2 Amplitude of single photo-electron

In case of a $1\times 10^7$ working gain, the pulse amplitude of SPE events is picked out from the recorded waveforms as described in Fig. 10a, and corresponds to the minimum value of the negative SPE pulse form. Typical values for each PMT are shown in Fig. 14b; HPK dynode PMTs have a slightly smaller SPE amplitude of about 6.5 mV compared to about 7.5 mV for the NNVT MCP-PMT, whereas HPK PMTs have a narrow distribution (with a standard deviation (STD) of 0.4 mV) compared to NNVT PMTs (STD of 2.5 mV). The different mean values of SPE amplitudes of the two PMT types are mainly influenced by the different pulse shapes related to the PMTs itself and the designs of the HV dividers.

4.3.3 Rise-time, fall-time and full width at half maximum of SPE pulses

Rise-time, fall-time and full width at half maximum (FWHM) of a typical pulse can be extracted from the full sample of recorded waveforms of each PMT, following the definition illustrated in Fig. 10a. As commonly known, the rise-time and fall-time are more related to the PMT itself, i.e. using the JUNO optimized version of the HV divider for the NNVT tubes, the rise-time was observed to be slower than in the case of using an early design version of the HV divider (which was used also for early testings). All qualified PMTs were tested in the containers with different pluggable JUNO optimized HV dividers [65,66,67]. Their SPE pulses show typical values for rise-time, fall-time, and FWHM of 6.9 ns, 10.2 ns and 11.6 ns for HPK PMTs, and 4.9 ns, 17.3 ns, and 7.9 ns for NNVT PMTs, respectively.

4.3.4 Charge resolution

The charge resolution of the PMTs’ SPE response is calculated using Eq. 5, where the $\sigma _{\textit{sig}}$ originates from the $\sigma$ of the Gaussian fitting of the PMT’s SPE charge spectrum as introduced in Sect. 4.2.1. The distribution of the SPE charge resolution for the full PMT sample as measured by the container system is shown in Fig. 15a, where the mean values are around 27.9$\%$ for the HPK PMTs and around 33.2$\%$ for the NNVT PMTs. However, the characteristic PMT charge response does not necessarily follow a Gaussian distribution [45], especially in case of the NNVT PMTs which normally show a long tail in their charge spectrum [24]. Nevertheless, the use of a Gaussian fit is justified since the parameter is used only as a relative control for charge response spread. The SPE charge resolution is required to be less than 40% for all individual PMTs of the JUNO CD, while it releases for a few NNVT tubes for the JUNO VETO detector:

$$\begin{aligned} \textit{Res.}\,(\%)=\frac{\sigma _{\textit{sig}}}{Q_{\textit{sig}}-Q_{\textit{ped}}}\times 100. \end{aligned}$$

(5)

Fig. 15

Left: SPE charge resolution of all qualified PMTs; right: measured P/V of all qualified PMTs. Black: all PMTs; red: NNVT; blue: HPK. Please note that the requirement of the charge resolution is relaxed for a few NNVT tubes for the JUNO veto detector, which is higher than 40%

4.3.5 Peak-to-valley ratio (P/V)

The peak-to-valley ratio (P/V) is a parameter to estimate the PMT’s ability to distinguish between noise and photo-electron signals which can be calculated using Eq. 6 and is based on the measured SPE charge spectrum (see Fig. 10b), where the valley $N_v$ is defined as the level of the local minimum between pedestal and SPE peak and is fitted by a parabolic function, while the peak $N_p$ describes the level of the SPE peak picked out from their Gaussian fit. The measurement results from the container systems of all qualified PMTs are shown on the right of Fig. 15b, where its typical values are around 3.8 for HPK PMTs and 3.9 for NNVT PMTs. The mean of the P/V of both PMT types satisfies the defined acceptance criteria (larger than 3 of HPK PMT and 3.5 of NNVT PMT, respectively), while some of individual PMTs show much smaller values $(< 2.5),$ especially the NNVT PMTs:

$$\begin{aligned} P/V=\frac{N_p}{N_v}. \end{aligned}$$

(6)

4.3.6 Excess noise factor (ENF) and gain excess normal distribution factor (gENF)

A previous study [45] also mentioned an excess noise factor (ENF), defined in Eq. 7 in order to effectively represent the measured charge spread. The larger the ENF under a similar light intensity, the broader the distribution of output signals will be for the same input.¹⁴$N_{\textit{pe}}$, $\sigma _{\textit{spec}}$ and $M_{\textit{spec}}$ in Eq. 7 are the measured number of p.e.s (mean number of p.e. from a measurement as µ in Eq. 1), the standard deviation and the mean of the whole charge spectrum distribution including the pedestal,¹⁵ respectively. The ENF distribution of all qualified PMTs is shown in Fig. 16a, where the typical values are 1.17 for HPK PMTs and 1.55 for NNVT PMTs. The distribution indicates that the variation of output signals of NNVT PMTs for the same input is much larger than in case of HPK PMTs:

$$\begin{aligned} \textit{ENF}=N_{\textit{pe}}~\times ~\left( \frac{\sigma _{\textit{spec}}}{M_{\textit{spec}}}\right) ^2. \end{aligned}$$

(7)

The gain excess normal distribution factor (gENF), as defined in Eq. 8 and discussed in [24, 85], is further characterizing the mismatch between the determined charge in p.e. at a gain G (determined from the SPE peak based on a normal distribution as in Eq. 2) and the expected charge based on the Poisson distribution (as in Eq. 1), where $Q_{\textit{pC}}$, $Q_{\textit{pe}}^{\textit{G}}$ and $Q_{\textit{pe}}^{\textit{Poisson}}$ are the measured charge¹⁶ of the PMT pulses in pC, the charge in p.e. expected for a gain G and the individually expected charge of a PMT pulse in p.e. based on Poisson statistics. $Q_{\textit{pe}}^{\textit{Poisson}}$ is only defined with a pulsed light source (such as the used LED) taken under a synchronized external trigger. Using the gENF factor, the expected charge in p.e. of a PMT measurement can be calculated as defined Eq. 9 for a spectrum on average or for a single signal:

$$\begin{aligned} \textit{gENF}= & {} \frac{Q_{\textit{pe}}^{G}}{Q_{\textit{pe}}^{\textit{Poisson}}} =\frac{Q_{\textit{pC}}/G}{Q_{\textit{pe}}^{\textit{Poisson}}} \end{aligned}$$

(8)

$$\begin{aligned} Q_{\textit{pe}}^{\textit{exp}}= & {} \frac{1}{\textit{gENF}}~\times ~Q_{\textit{pe}}^{G}. \end{aligned}$$

(9)

Fig. 16

Left: ENF distribution of all qualified bare PMTs. Right: gENF distribution of all qualified bare PMTs. Red: NNVT; blue: HPK

With the container systems, the gENF factor was measured and calculated for all qualified PMTs as shown in Fig. 16b. One can find that the gENF factor of the HPK dynode PMTs is smaller than 1 (0.93), while that of NNVT MCP-PMTs is bigger than 1 (1.28). The expected charge in p.e. is little larger than the calculated charge at a gain G for HPK dynode PMTs, while it is smaller for NNVT MCP-PMTs. This effect is caused mainly by the imperfect amplification process and correlated noise of the dynode PMT on one hand, and by the long tail of the charge spectrum which is typical for MCP-PMTs on the other hand (see the SPE amplitude spectra in Sect. 4.5.1); those effects are discussed in more details also in [24].

4.4 Photon detection efficiency (PDE)

Instead of setting a requirement on the QE of the PMTs, JUNO uses the photon detection efficiency (PDE) to measure the sensitivity of the PMTs. Here, the PDE of the 20-inch PMTs is defined as the photon detection efficiency for light impinging vertically on the PMT surface averaged over the whole photocathode area with a surface area weight. This can be measured directly by the scanning station by calculating a surface weighted average of all its measurement points across the PMT area¹⁷ considering the geometry differences of HPK and NNVT PMTs, where a reference PMT (HPK R1355 as mentioned in Sect. 3.1.2) is used for normalization. Following the QE measurements for samples of both types of 20-inch PMTs in [19, 22, 83, 86,87,88], a special comparison study on the relative collection efficiency among PMTs with different collection structure [25] has concluded that the reference PMT’s QE can be dealt as its PDE, assuming a $100\%$ collection efficiency in case of a 5 mm light spot impinging on the PMT center. Furthermore, the measured PDE of the container systems is normalized drawer by drawer to the scanning station using a set of 20-inch PMTs that have been measured in both the container and scanning station systems. This set of 20-inch PMTs with known PDE was also used to individually calibrate the light intensities in the drawers to enable a reliable PDE measurement in the first place.

4.4.1 PDE measurement

The PDE measurements of the containers and scanning stations are realized by employing the photon counting method on the charge spectra obtained with the stabilized pulsed LEDs operated at low intensity, where the effects of threshold and dark count rate are checked and considered as uncertainty. Considering the observed noise level and charge resolution, a threshold of around 0.25 p.e. was chosen to separate the pedestal from photon induced events. The light intensity used in both container and scanning station system is a low multi-photon level (with 1–2 p.e.s observed by the PMTs) to minimize the uncertainty from system and statistics as suggested in [72]. Based on the repeatability of the daily measurements of the monitoring PMTs (see Sect. 3.1.1), a measurement uncertainty of better than 1% of the absolute PDE (3% relative PDE)¹⁸ is achieved for both containers [69] as well as for both scanning stations (see Sect. 3.1.2).

The PDE distribution of all qualified PMTs measured by the container system is shown in Fig. 17a, where the mean PDE is $28.1\%$ for HPK PMTs, and $28.9\%$ for NNVT PMTs. The sample of qualified NNVT PMTs can be classified as 4609 NNVT low-QE PMTs (early version of the MCP-PMTs and identified by production dates) with a mean PDE of $26.8\%$, and 10,456 NNVT high-QE PMTs (latest version of the MCP-PMTs with higher QE, again identified by production dates) with a mean PDE of $29.9\%$ (Fig. 17b). Following the proposed selection to distribute the qualified PMTs between JUNO CD and veto detector, the mean PDE for the 17,612 PMTs selected for the CD is $29.1\%$ (including all HPK PMTs¹⁹), while the mean PDE for the 2400 PMTs selected to be used in the veto detector is $25.6\%$.

Fig. 17

Measured PDE distribution of all qualified PMTs (left), and of all qualified NNVT PMTs (right). Black: all PMTs; red: NNVT; blue: HPK; Right: dark green: High_QE; deeppink: Low_QE

4.4.2 Light system aging and PDE correction

Within the $\sim 4$ years of operation of the two container systems, a decrease in the measured PDE of the monitoring PMTs was found for both of container $\#$A and $\#$B as shown on top of Fig. 18. Both HPK and NNVT monitoring PMTs show a similar trend, which however required a long period of observation to be confirmed within the stated PDE uncertainty. The results were checked in detail based on the results of the monitoring PMTs as well as based on additional cross-checks with additional PMTs on the consistency among drawers, between the two containers, between the containers and scanning stations, as well as between the scanning stations themselves. It is concludes that the decrease is mainly caused by the LED light system²⁰ itself used in the containers rather than by direct PMT aging. The effect directly depends on the container drawer as well. The scanning stations do not observe such a decreasing effect in the same period, according to the calibration and monitoring by the reference PMTs.

Fig. 18

Relative variation of the measured PDE of the monitoring PMTs (HPK PMTs tagged by “EA”, NNVT PMTs tagged by “PA”) at the container system, before and after a correction based on a recalibration at the end of the regular testing period. Indicated variation is relative to the beginning of the testing period, which is used to compare among the PMTs. Top: directly measured PDE (with top left: container $\#$A; top right: container $\#$B). Bottom: PDE after aging correction (with bottom left: container $\#$A; bottom right: container $\#$B)

Based on a dedicated recalibration of all individual container channels at the end of the regular testing campaign and the observations of the monitoring PMTs from all individual drawers, a correction factor for the measured PDE was specified and applied for each drawer box of both containers, using a linear assumption for the PDE decrease relative to the testing date. The correction scale on average of all drawers was found to be $6.5 \times 10^{-5}$/day for container $\#$A and $\sim 7.6 \times 10^{-5}$/day for container $\#$B and was applied for both HPK and NNVT PMTs. Finally, the corrected PDE results show a stable behaviour of the monitoring PMTs in both container systems, as well as consistent results with the scanning stations and between both containers.

The largest residual factor after the aging correction of all drawers is considered as the final bias uncertainty of the container system, which is about 2.3% relatively for the tubes tested after 1000 days with an aging factor of $2.3\times 10^{-5}$/day, translating to an absolute value of 0.7% for tubes with 30% PDE. After applying the aging correction of each drawer to the measured PDE of the container systems, the distributions of the corrected PDE results from the measurements of the monitoring PMTs (as shown at the bottom of Fig. 18) are checked again as an uncertainty control. The variations are now within the aimed uncertainty range of $\lesssim 1\%$ absolute PDE, in particular when taking the spread of the corrected PDE distribution of each monitoring PMT into account. The largest variation of container #B is provided by the fixed drawer monitoring PMT that is not used for daily measurements, which also includes a possible variation of the PMT itself.

The distributions of the corrected PDE results of all qualified PMTs are updated and depicted in Fig. 19a, where the averaged values now increased to $28.5\%$ for the HPK PMTs and $30.1\%$ for NNVT PMTs. The average PDE of the qualified NNVT low-QE PMTs was updated to $27.3\%$, and $31.3\%$ for the qualified NNVT high-QE PMTs respectively,²¹ as shown in Fig. 19b. Furthermore, the corrected results of the proposed PMTs for JUNO CD and veto detector are also updated, with the averaged PDE increase to $30.1\%$ in the CD and to $26.1\%$ in the veto detector.

Fig. 19

Corrected PDE distribution of all qualified PMTs (left), and of all NNVT PMTs (right). Black: all PMTs; red: NNVT; blue: HPK; dark green: high-QE MCP-PMTs; deep pink: low-QE MCP-PMTs

4.5 Dark count rate (DCR)

Using the container system, the dark count rate (DCR) of each individual PMT was measured under a $1\times 10^7$ gain. Data acquisition was performed using a CAEN V830AC scaler together with a CAEN V895B leading-edge discriminator, as described in [69]. The measured DCR value can be affected by the applied signal threshold, the time the PMT initially stayed in darkness in order to stabilize its dark rate prior to the measurement (“cooling time”), the photocathode’s temperature, and the applied gain. The effect of each factor was checked in detail and will be discussed in the following.

4.5.1 Amplitude threshold

A quarter p.e. threshold was selected to consider the noise level of the system, the SPE charge resolution of the PMTs, as well as possible future JUNO conditions. The target threshold in charge corresponds to a signal threshold of about 3 mV for the pulse amplitude of both HPK and NNVT PMTs. The optimal threshold is not located precisely at the same level for both PMT types due to pulse shape differences between HPK and NNVT PMTs – moreover, it is expected to be specific for each PMT rather than only refer to the mean amplitude value for an SPE pulse at a gain of $1\times 10^7$. Since there are operational constraints set by the electronics used in the test,²² a threshold of 3 mV for the pulse amplitude was selected for the DCR measurements and was found to be a good compromise between measurement accuracy and comparability on one side, and mentioned constraints and present noise level on the other side, as illustrated in Fig. 20.

Fig. 20

Effect of the set amplitude threshold on the DCR measurement of a single, typical PMT. The cut distributions (in blue) represent all pulses with a charge above the threshold of 0.25 p.e., matching well to a pulse amplitude threshold of 3 mV. Left: HPK pulse amplitudes; right: NNVT pulse amplitudes

As indicated in Fig. 6, PMTs failing the DCR acceptance criteria in the containers have been reevaluated in another container run or finally using the scanning station to exclude any other possible uncertainties on threshold or noise level. Since the scanning stations are using a factor $\times 10$ amplifier prior to the counting electronics (see Sect. 3.1.2 again), they can apply i.e. an adequate DCR threshold with much higher precision than possible in the container systems.

4.5.2 Cooling time

Large area PMTs such as the used 20-inch PMTs of JUNO generally need several hours in a dark environment to stabilize their dark rate after applying the HV. This is valid i.e. after the PMT was exposed to ambient light. As discussed in [69], all PMTs tested in the container system are resting for at least 12 h in darkness before the start of the PMT characterization to ensure an effective (and significant) DCR measurement.²³ During this time, the DCR will be frequently monitored as exemplarily presented in Fig. 21. The concluding DCR measurement is usually performed after about 16 h in darkness, as explained in [69].²⁴ The gain has been set to a level of $1\times 10^7$ following the procedure described in Sect. 4.2.1 prior to this measurement.

4.5.3 Temperature

As discussed in [19, 27,28,29], the actual DCR of a PMT is a function of temperature. We thereby assume that the photocathode’s temperature is in equilibrium with the temperature of the surrounding air, i.e. after a several hours stay in the drawers. To estimate the effect of different air temperatures on the DCR results of the JUNO PMTs during their characterizations, data from the temperature monitoring system within the containers was taken into account to the observed DCR of the PMTs. This was done for a large PMT sub-sample containing more than 1800 HPK PMTs and more than 11,900 NNVT PMTs. Although every PMT acts uniquely, a slight trend could be observed for each PMT type, indicating a mild temperature dependence of about 0.5 kHz/$^{\circ }$C for the combined HPK PMT sub-sample, and of about 3.0 kHz/$^{\circ }$C for the combined NNVT PMTs sub-sample within a temperature range of 19–29$^{\circ }$C as present during the mass characterization campaign, see also Fig. 22.

Fig. 21

Measured DCR vs. cooling time for a typical HPK and NNVT PMT. The measured DCR values are normalized to the value taken after 12 h (time after loading the PMT into the container). The indicated time of $\sim 16$ h marks the usual time when the final DCR measurement is performed as part of the regular PMT characterization process

Additionally, specific temperature surveys were performed using the container system with its HVAC unit to gain a better understanding of the temperature dependence of the measured DCR. In these surveys, the effect of temperature to the measured DCR was investigated by varying the temperature in a range from 14 to 28 $^{\circ }$C (with a rate of 0.5 $^{\circ }$C/h) and monitoring the DCR in parallel. Such a testing cycle containing 26 HPK and 26 NNVT PMTs is shown in Fig. 23, where the DCR of every measurement was normalized to the value at 22 $^\circ$C of each PMT individually. The results of these surveys are confirming a stronger temperature dependence of the NNVT PMTs than for HPK PMTs, which is on average about 0.2 kHz/$^{\circ }$C ($\sim 2\%/^{\circ }$C relative DCR change) for the combined HPK PMT sample, and about 4.8 kHz/$^{\circ }$C ($\sim 10\%/^{\circ }$C relative DCR change) for the combined NNVT PMT sample. These values are consistent with the before mentioned estimation between 19 and 28 $^{\circ }$C in a first order approx. The results further indicate, that the observed DCR is significantly increasing for temperatures higher than 22$^\circ$C (particularly for the NNVT PMTs). It should also be noted, although not visible in the plot, that PMTs showing a higher absolute DCR at 22 $^\circ$C are more sensitive to temperature changes, compared to PMTs with a lower absolute DCR at 22 $^\circ$C.

Fig. 22

Observed correlation between air temperature in the container and measured DCR of a large sub-sample of PMTs. The black dots indicate the mean and STD of all DCR measurements within the corresponding $0.25\,^\circ$C slice

Fig. 23

Measured DCR versus temperature in a specific temperature survey. The black dots represent the averaged trend of all measured PMTs under the same temperature. The discrete points far from the trend are generated by noise during the tests

4.5.4 Measured dark count rate (DCR)

The measured DCR distributions (as shown in Fig. 24) of all qualified PMTs were tested by the container systems using the discussed amplitude threshold of 3 mV (p.e. threshold of 0.25, refer to Fig. 20), with a cooling time of $\sim 16$ h, and at an average air temperature of $25 \pm 3~^{\circ }$C. The mean value is around 15.3 kHz for HPK PMTs and 49.3 kHz for NNVT PMTs. The NNVT DCR distribution shows a long tail which is cut at 100 kHz according to the requirement of JUNO as listed in Table 1. Also, both low- and high-QE MCP-PMTs show a consistent DCR distribution. In the final JUNO detector, an even lower average DCR is expected based on the considerations in Sect. 4.5.3, since the water temperature in the veto pool (where the PMTs will be embedded in) is predicted to be at about $21 \pm 1^{\circ }$C.

Fig. 24

Measured DCR distribution of qualified PMTs. Left: black solid: all PMTs; red solid: NNVT; blue solid: HPK; Right: dark green solid: High_QE; deep pink solid: Low_QE. Note: the DCR of NNVT PMTs is selected with $\le$ 100 kHz

4.6 Transit time spread (TTS)

The transit time spread (TTS) describes the timing resolution of the PMT. It represents the spread of different photo-electron transit times within the PMT bulb, which depend on the interaction point and emission angle of the photo-electron, as well as on the energy of the released photo-electron and the focusing electric field distribution [90]. For JUNO, the timing resolution of the HPK dynode PMTs is crucial for a precise event reconstruction in the detector. For these PMTs, the distribution of transit times (TT)²⁵ follows a Gaussian distribution as shown in Fig. 25a, where the TTS is defined as the $\sigma$ of this distribution, with $\sigma \simeq {\textrm{FWHM}} / (2 \sqrt{2 \ln 2})$ [28]. In case of the NNVT MCP-PMTs, this relative TT distribution is much more complex and contains a varying number of substructures (sub-peaks), see Fig. 25b. These features are related to the PMT design and MCP structures [86] and are a consequence of the optimization of the MCP-PMTs for a maximum collection efficiency, which leads to a worse timing resolution [91] (will be discussed in Sect. 4.10.4). As a consequence, the TTS can not be described using a Gaussian fit, but rather will be described here as the mean standard deviation of the TT distribution with the edges cut away (only the quantiles $\left[ Q_{03}:Q_{97}\right]$ are taken into account) to remove outliers and noise events.²⁶

Fig. 25

Typical relative transit time (TT) distributions for an HPK (left) and an NNVT PMT (right, indicated “hit times”). The PMT’s TTS is determined using a Gaussian fit in case of the HPK PMTs, and using the std. dev. of a cut distribution (use only quantiles $Q_{03}$ to $Q_{97}$) in case of the NNVT PMTs. Both methods produce consistent results if applied on the data of HPK PMTs

All valid TTS results discussed in this section are from measurements with the container system using the pico-second laser fiber system²⁷ [69, 74] and similar settings as used for the SPE features in Sect. 4.3 (i.e. a light intensity of µ$\simeq 0.1-1$ p.e.), but with larger statistics of up to 50,000 trigger events. The triggers for the data acquisition and light sources are provided by a Keysight 33512B arbitrary waveform generator featuring a jitter of only $\sim 50$ ps [92]. Unfortunately only container $\#$B has a sufficient timing resolution, with a minimum measurable TTS of $\sim 0.8$ ns (that’s the reason why there is a left cut around 0.8 ns in Fig. 26b) – due to a mechanical incident, the fiber system of container $\#$A can not provide a timing resolution of $<2$ ns anymore. The final accuracy for the TTS measurements achieved with container $\#$B’s laser is less than 0.2 ns (RMS 0.2 ns) for HPK PMTs and less than 0.6 ns (RMS 1.4 ns) for NNVT PMTs²⁸ (also see Fig. 26a). Systematic effects introduced by the data taking electronics and the fiber system were measured to be only about 0.6 ns on average for all channels and were corrected in order to achieve a fair estimation of the PMTs’ TTS.

Fig. 26

Left: TTS results of the monitoring PMTs in container #B. Combined result distribution of all monitoring PMTs (3 HPK, 2 NNVT, tested in all channels) shows a good reproducibility of the results. Right: TTS distribution of a sub-sample of qualified bare PMTs containing 837 HPK and 3610 NNVT PMTs, measured with the ps-Laser system (red: NNVT; blue: HPK)

Since only one of containers $\#$A and $\#$B provides a sufficient timing resolution, the TTS was evaluated only for a large sub-sample of all tested PMTs. The results of this sub-sample are representative for the full sample of qualified PMTs for JUNO as presented in the other sections. Results are available for a total of 837 HPK and 3610 NNVT PMT of qualified bare PMTs tested with the laser system in container $\#$B, covering about 16.7% of the full qualified HPK PMT sample, and about 24.0% of the full qualified NNVT PMT sample.²⁹ The results’ distribution is depicted in Fig. 26b. The mean TTS is about 1.3 ns in $\sigma$ as required for HPK dynode PMTs, with only about $1\%$ of the examined PMTs clearly failing the timing performance requirements for JUNO, which is at an acceptable level. Results are also consistent with another study performed earlier, see [93] for details. The mean TTS of NNVT MCP-PMTs is about 7.0 ns, see also Fig. 26b.

Fig. 27

Relative TT distribution from LED measurements of single PMT and the relative TTS distribution of sub-sample of all qualified bare PMTs measured with LED in the container system. Dark-green, NNVT PMT; blue, HPK PMT

Fig. 28

Pre-pulse ratio of all qualified PMTs. Black: all PMTs; red: NNVT PMTs; blue: HPK PMTs

The relative transit times’ distribution can be calculated also from the data shown in Sects. 4.3.2 and 4.3.3 using the LED system of the containers (SPE light intensity, 20,000 waveforms).³⁰ This is shown in Fig. 27a for an example, single HPK and NNVT PMT. Comparing to the results shown in Fig. 25b using the laser system, the sub-peaks in the relative TT distribution of NNVT MCP-PMTs are smeared out by the container LEDs – this will be further discussed in Sect. 4.10.4 in the scope of the scanning station (which also uses LEDs). When calculating the relative TTS of each PMT measured with the LED system of the containers,³¹ the relative TTS can be described by the sigma of a directly Gaussian fit on the relative TT distribution, as shown in Fig. 27a. Using the monitoring PMTs for checking the repeatability of these measurements, the final accuracy for the relative TTS measurements using the LEDs achieved with containers $\#$A and $\#$B is around $\sigma \sim 0.2$ ns (RMS 1.9 ns) for HPK PMTs and around $\sigma \sim 0.7$ ns (RMS 1.6 ns) for NNVT PMTs, which is consistent to the laser measurements. The typical values of the relative TTS from the LED measurements for all qualified PMTs is 2.6 ns for HPK and 8.4 ns for NNVT PMTs, as shown also in Fig. 27b. The results for both PMT types are larger than the values acquired from the laser measurements in container $\#$B as expected from the smearing due to the LED pulses and differences of the photon distribution on the photocathode between laser and LED system. Therefore, the results acquired using the LED system will be used only for a relative comparison and systematic control, and for comparison with the results of the scanning stations (will be presented in Sect. 4.10.4).

4.7 Pre-pulse and after-pulse ratio

Pre-pulses originate mainly from photons which hit the first dynode or MCP directly rather than the photocathode. Hence, they have a smaller pulse amplitude than the main pulses and show up before the main pulse by a few to tens of nanoseconds. With the recorded waveform from the container systems, a charge ratio is checked between the pre-pulse and main pulse in a window of $[-80,-10]$ ns before the peak of the main pulse. The general results are shown in Fig. 28. The pre-pulse ratio is $<1\%$ for HPK PMTs and around 1% for NNVT PMTs, while there is a larger uncertainty for NNVT MCP-PMTs due to their larger TTS smearing.

Fig. 29

Typical time features of after-pulses for two typical PMTs. Top: NNVT; bottom, HPK (Hamamatsu)

Fig. 30

The after-pulse intensity in p.e. versus time after the initial pulse. Left, HPK; Right, NNVT. The z-axis (color palette) presents the entries of the pulses with the same charge versus time

Another possible issue of PMTs is the contamination of the vacuum bulb with gases. Molecules inside the PMT glass envelope can be ionized when the photo-electrons are passing through. These ions will travel back and hit the photocathode, ejecting more electrons. Such events will cause an after-pulse, which features a charge ratio that is proportional to the initial pulse while the after-pulse will be delayed in time by hundreds of nanoseconds to tens of microseconds, depending on the gas molecule, the dimension of the glass bulb and the strength of the applied electric field. The after-pulse ratio of 20-inch PMTs is calculated in a window from 500 to 20,000 ns after an initial pulse of about 100 p.e. with a special LED configuration. This ratio is expected to be less than 15% of the initial pulse charge.

There are already distinct studies about after-pulses [22, 28, 48, 94, 95]. This study only measured a sample of several $\sim 100~\hbox {s}$ PMTs using the scanning station as shown in [96]. The mean after-pulse charge ratio is calculated from 150 NNVT MCP-PMTs and 7 HPK dynode PMTs respectively. This sample leads to the result of 6.7$\%$ for NNVT MCP-PMTs and 12.0$\%$ for HPK dynode PMTs. At the same time, it was found that the HPK dynode PMTs and NNVT MCP-PMTs have very different after-pulse features, as indicated for an example PMT shown in Fig. 29: the selected MCP-PMT from NNVT was measured with a $\sim 7.0\%$ after-pulse charge ratio, showing four typical after-pulse peaks in time around 0.8 µs (ratio 3.1$\%$), 3.2 µs (ratio 1.4$\%$), 4.6 µs (ratio 1.7$\%$) and 17.0 µs(ratio 0.8$\%$, only a hint), while the selected HPK dynode PMT was measured with an after-pulse charge ratio of 13.8% and typical peaks at 0.8 µs (ratio 1.1$\%$), 3.9 µs (ratio 8.7$\%$) and 14.2 µs (ratio 4.0$\%$). The intensity in charge of a single after-pulse is mainly at the SPE level, while it can reach up to several O(10) p.e.s as shown in Fig. 30: so in the case of this particular HPK PMT, the charge ratio of the after-pulses at 3.9 µs is larger than the pulse at 14.2 µs despite the count rates are vice versa (compare Fig. 29 again).

4.8 Non-linearity

Neutrino events detected in JUNO typically correspond to a signal strength of up to few hundreds of p.e.s in each of the 20-inch PMTs. Furthermore, there are huge signals expected from physics or background events with much higher energy such as cosmic muons. A dynamic range in linearity of the PMT response with the designed HV divider is required to reach 1000 p.e.s with less than 10$\%$ distortion. The HV divider design for the 20-inch PMTs of JUNO described in [18, 65,66,67] was designed to provide a DC of $\sim 100$ µA for HPK PMTs and $\sim 180$ µA for NNVT PMTs at a working gain of $1\times 10^7$. A few samples of both PMT types were measured by the double-LED (LED A, LED B, C is when LED A and B flashing at the same time) method (C/(A+B)) [23] in a pulsed mode, and a cross-checking method between the 20-inch PMT and the 3-inch PMT. One resulting curve of a NNVT MCP-PMT is shown in Fig. 31, where the PMT response with the designed HV divider can satisfy the requirements, and the results of both methods are consistent. The current design can satisfy the requirement on the linearity response. Another standalone paper will be prepared on this topic.

Fig. 31

Measured linearity of a 20-inch PMT by double-LED method (C/(A+B)) (red rectangular) and cross checking with 3-inch PMT (SPMT monitored) (blue triangle)

4.9 Earth magnetic field (EMF) effect

PMTs with large vacuum bulbs are significantly affected by magnetic fields when the photoelectron drifts to the collection dynode or MCP, in fact, their performance is even sensitive to the Earth magnetic field (EMF, $\sim 50$ µT at the JUNO’s experimental site). This requires a more thorough characterization on the magnetic field effect to PMTs. Using the scanning station, the EMF’s strength around PMT’s location can be surveyed in the range $-50$ µT to $+50$ µT by changing the DC configuration of the Helmholtz coils in the housing dark rooms. As shown in Fig. 32, the variation of the performance of PMTs is negligible $(< 1\%\sim 3\%)$ when the residual field is less than 10$\%$ of EMF. The direction of the magnetic field is perpendicular to the central axis of the PMT, which is considered as the worst condition, and followed by the direction of the collection box of dynode PMTs or the direction of micro-channel of MCP-PMT. The container system and the scanning stations are operating with the magnetic field suppressed to a level of $<5$ µT. Following these measurements, the requirement on the residual magnetic field in the JUNO detector is specified to be $\lesssim 5$ µT. As one can see, NNVT PMTs are affected more strongly than HPK PMTs, which could be explained by differences in the photoelectron focusing process. For more details, see [97].

Fig. 32

Averaged PMT PDE versus remaining magnetic field (MF) strength tested with 9 HPK and 15 NNVT PMTs

4.10 Uniformity

For a 20-inch PMT, the anode uniformities along the full photocathode are significant for a better understanding of the PMT itself and the detector’s response. An exemplary PMT surface scan performed by the scanning station is shown in Fig. 33 with 15$^{\circ }$ steps along the azimuthal angle ($\phi$) by the seven LEDs distributed in zenith angle ($\theta$), the online integrated charge spectrum of one spot, and the scanned map on PDE and gain. Thanks to the powerful capability of the scanning stations, detailed studies were made using a few thousand PMTs (585 HPK PMTs, 2658 NNVT PMTs, including 939 high-QE PMTs, and 1719 low-QE PMTs) to characterize the uniformity, which will be discussed in detail as follows.

Fig. 33

Exemplary data of a PMT scan with the scanning station. Top left: scanning process indication; top right: integrated charge spectrum of selected area element; bottom: measured maps of an NNVT MCP-PMT (PA2004-1007) on PDE (bottom left) and gain (bottom right)

4.10.1 Uniformity of gain and gain excess normal distribution factor (gENF)

The charge response along the whole photocathode surface can be described firstly by its gain. As known, there will be a large response deformation around the equator ($\theta \sim 90^\circ$) of the glass bulb, compared to the top pole ($\theta \sim 0^\circ$) of the photocathode. Following the gain definition in Sect. 4.2.1 and applying the determined HV for a gain of $1\times 10^{7}$, the gain is determined for each LED spot measurement individually. The uniformity along the whole photocathode in the zenith ($\theta$) and azimuthal ($\phi$) angles is shown in Fig. 34: each point is the average of all the measurements of the full, tested PMT sample with the same zenith angle or azimuthal angle individually, and the uncertainty bar in the vertical axis is calculated from the error of the average. The gain of HPK PMTs is more uniform for $\theta <70^\circ$ along the zenith angle, but it decreases sharply by $\sim 30\%$ towards the equator. Along the azimuthal angle, the gain of HPK PMTs shows systematically smaller values $(\sim 5\%)$ and some sub-structures (peak-to-peak variation $\sim 10\%)$ around its collection box in a rectangle shape, including the peaks before ($\phi \sim 200^\circ$), around $\phi \sim 270^\circ$, and $\phi \sim 0^\circ$ in particular. On the other hand, the gain of NNVT PMTs is continuously increasing as the zenith angle increases up to $\theta \sim 70^\circ$ before drops at the equator with a total variation range of $\sim 10\%.$ Along the azimuthal angle ($\phi$), the NNVT PMTs are more uniform than the HPK PMTs and one can observe a two-cycles-oscillation with the valleys located at around $\phi \sim 50^\circ$ and $\phi \sim 230^\circ$ with a peak-to-peak variation of $\sim 3\%,$ which should be related to the direction of the MCP channels. The averaged gain of the full photocathode of all PMTs tested by the scanning station is $1.07\times 10^{7}$ for HPK PMTs and $1.09\times 10^{7}$ for NNVT PMTs, and they are systematically higher than the gain measured by the container ($1.00\times 10^7$ for HPK PMTs and $1.03\times 10^7$ for NNVT PMTs, respectively). The difference is assumed from the amplification factor of the stations, which is not exactly equal to the claimed $\times 10.$

Fig. 34

Uniformity of gain along the zenith angle ($\theta$, left) and azimuthal angle ($\phi$, right). HPK: blue; NNVT: red

As a correlated factor of the gain, the uniformity of gENF (defined in Sect. 4.3.6) is also checked with the same data set as the gain uniformity, which is shown in Fig. 35 in the zenith ($\theta$) and azimuthal angles ($\phi$). Firstly, the absolute value difference between HPK and NNVT PMTs is related to the PMT itself as discussed in Sect. 4.3.6. The gENF on average of the whole photocathode of the full, scanned sample is 0.89 for HPK PMTs and 1.38 for NNVT PMTs. Comparing to the values acquired by the containers, the discrepancy is mainly assumed from the averaging scheme and the contribution of the gENF at larger $\theta$. Except the absolute value difference, the gENF of both PMT types is more uniform along the zenith angle (when $\theta <80^\circ$) and azimuthal angle. At the same time, there are a few sub-structures worth mentioning:

• (1) The gENF value shows a larger change for $\theta$>80$^\circ$.

• (2) NNVT PMTs show a variation $(\sim 6\%)$ sub-structure for $\theta <80^\circ$ (peak at $\theta \sim 50^\circ$ and valley at $\theta \sim 70^\circ$).

• (3) The gENF factor of HPK PMTs along the azimuthal angle shows a small structure $(\sim 4\%),$ which could be related to that in Fig. 34.

• (4) The gENF factor of NNVT PMTs along the azimuthal angle shows a two-cycles-oscillation (variation $\sim 1.5\%)$ at its gain but with opposite phases (peaks at $\phi \sim 60^\circ$ and $\phi \sim 210^\circ$).

Fig. 35

Uniformity of gENF along the zenith angle ($\theta$, left) and azimuthal angle ($\phi$, right). HPK: blue; NNVT: red

4.10.2 Uniformity of resolution and excess noise factor (ENF)

Following the SPE charge resolution definition in Sect. 4.3.4, the uniformity of the SPE charge resolution on the whole photocathode is shown in Fig. 36 in the zenith ($\theta$) and azimuthal ($\phi$) angles. Each point is directly averaged from all PMTs tested by the scanning station with the same zenith angle or azimuthal angle individually. The total mean charge resolution of the whole photocathode and for all the scanned PMTs is 0.28 for HPK and 0.32 for NNVT. Comparing the uniformity results of HPK and NNVT PMTs, the charge resolution is almost constant when $\theta <70^\circ$ but getting larger towards to the equator for both of them (but in particular for HPK PMTs). Along the azimuthal angle, the charge resolution of both HPK and NNVT PMTs shows a similar trend but in opposite phase at the gain as shown on the right of Fig. 34: it is systematically higher than average around $\phi \sim$ 270$^\circ$ and small fluctuations can be observed (variation $\sim$ 14%) for HPK PMTs and a two-cycles-oscillation (variation $\sim$ 10%) for the NNVT PMTs with peaks located at around $\phi \sim$ 50$^\circ$ and $\phi \sim$ 230$^\circ$.

Fig. 36

Uniformity of the SPE charge resolution along the zenith angle ($\theta$, left) and azimuthal angle ($\phi$, right). HPK: blue; NNVT: red

As a correlated factor to the SPE charge resolution, the uniformity of ENF (defined in Sect. 4.3.4) along the whole photocathode is determined for the same data set as done for the charge resolution analysis, see Fig. 37. The ENF on average over the whole photocathode of all scanned PMTs is 1.08 for HPK PMTs and 1.41 for NNVT PMTs. The HPK PMTs show a more uniform performance along both zenith angle and azimuthal angles. The NNVT PMTs reach $\sim$ 1.5% as the zenith angle increases and also show a two-cycles-oscillation with a variation of $\sim$ 1.5% along the azimuthal angle with the peaks located at around $\phi \sim$ 50$^\circ$ and $\phi \sim$ 230$^\circ$. The NNVT PMTs show a similar trend as in case of the gENF factor and charge resolution along the zenith angle to a max. change of $\sim$ 18%, while a clear dependence to azimuthal angles also with a two-cycles-oscillation with peak-to-peak variation of $\sim$ 2% peaks located at around $\phi \sim$ 50$^\circ$ and $\phi \sim$ 230$^\circ$.

Fig. 37

Uniformity of ENF along the zenith angle ($\theta$, left) and azimuthal angle ($\phi$, right). HPK: blue; NNVT: red

4.10.3 Uniformity of photon detection efficiency (PDE)

The measurement of the PDE uniformity along the whole photocathode of all scanned PMTs is shown in Fig. 38 and divided in three data sets: HPK (Hamamatsu), High-QE NNVT (HiQE) and Low-QE NNVT (NNVT). The PDE along the zenith ($\theta$) and azimuthal ($\phi$) angle is the averaged value of all the measured spots with the same zenith and azimuthal angle of all the scanned PMTs. The high-QE NNVT PMTs show a better performance than HPK PMTs in both directions: a better uniformity towards the equator and a larger absolute value in both zenith and azimuthal direction. The low-QE NNVT PMTs show a smaller PDE which has slightly better uniformity than for the high-QE PMTs along $\theta$. The HPK PMTs’ PDE sharply drops from $\theta \sim$ 80$^\circ$ to $\theta \sim$ 90$^\circ$ by about 40%, and a clear asymmetry (max. $\sim$ 10%) is observed along azimuthal angle related to its focusing box direction around $\phi \sim$ 270$^\circ$. Both the low-QE and high-QE NNVT PMTs show a two-cycles-oscillation along azimuthal angle (peak-to-peak variation is $\sim$ 3%) and peaks located at around $\phi \sim$ 110$^\circ$ and $\phi \sim$ 300$^\circ$.

Following the requirements, the non-uniformity of PDE (defined as the ratio of the distribution of the PDE of each light spot to their average) should not exceed 15%. In Fig. 39, the distributions for 320 NNVT and 245 HPK randomly selected PMTs are presented. These samples allow an estimation of the total number of PMTs failing the requirements as (0.5–1.7)% at CL = 68% for NNVT and (0.0–0.5)% at CL = 68% for HPK PMTs. Considering such a small number of non-uniform PMTs for the general sample there is no strict necessity to test all the PMTs on the scanning system and no PMT is rejected by the non-uniformity. These distributions will be useful for further detector simulations to evaluate the PMTs’ non-uniformity contribution to the JUNO energy resolution.

Fig. 38

Uniformity of PDE along the zenith angle ($\theta$, left) and azimuthal angle ($\phi$, right). Blue,: HPK PMTs; red: NNVT low-QE PMTs; green: NNVT high-QE PMTs

Fig. 39

PDE non-uniformity distributions for randomly sampled PMTs. Black lines show the threshold for PDE non-uniformity (15%), numbers in brackets indicate failed PMTs. Left: HPK (Hamamatsu) PMTs; right: NNVT PMTs

4.10.4 Uniformity of relative transition time spread (TTS) and transition time (TT)

The scanning station only implements the stabilized-LED flashing in pulse mode. The LED flashing intensity detected by PMT is 1 $\sim$ 1.5 p.e. for the PDE measurement, where the timing resolution can only reach $\sigma \sim$ 2 ns to realize a relative TTS comparison along the photocathode  [71]. Figure 40 shows an example of the hit time (relative TT) distribution of a single HPK and NNVT PMT measurement analyzed with constant fraction discrimination (CFD). The NNVT PMT shows a broader distribution than the HPK PMT. Additionally, a sub-peak structure can be observed, that is similar to the right plot of Fig. 40. There is a clear dependence of the typical relative TT value and its distribution spread on the location of the LED spots as shown on the right of Fig. 40. We can distinguish some sub-peaks among LED_1, LED_3, and particularly LED_5: the sub-peak structure (as introduced before in Fig. 25b) of the NNVT PMT’s relative TT is related to the location of the photon hitting on the photocathode due to its relation on the focusing electric field and MCP itself.

Fig. 40

PMT hit time (relative TT) distribution of a single NNVT and HPK PMT directly measured in the scanning station with different LEDs (locations) at azimuthal angle $\phi$ = 0. Left: HPK (blue: 5th LED; orange: 7th LED); right: NNVT (blue: 1st LED; orange: 3rd LED; green: 5th LED)

A Gaussian fitting is applied to the relative TT spectrum (Fig. 40) around its maximum peak (±20 ns) on each light spot of the scanning station measurements, the fitted mean is treated as the relative TT and the $\sigma$ as the relative TTS. Figure 41 shows the average of the relative TTS of all scanned PMTs along the respective zenith or azimuthal angle. The relative TTS average over the whole photocathode of all scanned PMTs is 2.3 ns for HPK PMTs and 6.6 ns for NNVT PMTs, where the discrepancy to the results of the container LEDs is mainly due to the averaging scheme and ignoring the later discussed relative TT difference. HPK dynode PMTs show a better uniformity (max. variation $\sim$ 0.1 ns) along the zenith angle for $\theta <80^{\circ }$. On the other hand, the NNVT MCP-PMTs show a larger TTS than HPK PMTs and a clear zenith angle dependence: the larger $\theta$, the larger the relative TTS (relative TTS increases 0.25 ns/10$^{\circ }$ when $\theta <80^{\circ }$ on average with a linear assumption). The HPK PMTs also show some tiny structures along the azimuthal angle in the range of 0.15 ns and a single-cycle-oscillation related to its focusing box direction at $\phi \sim$ 270$^\circ$.

Fig. 41

Uniformity of the relative TTS ($\sigma$) along the zenith ($\theta$, left) and azimuthal angle ($\phi$, right) on average of all scanned PMTs with the same zenith or azimuthal angle. Blue: HPK PMTs; red: NNVT PMTs

At the same time, the uniformity of the transit time (TT) is also determined for both HPK and NNVT PMTs, which could also contribute to the TTS, i.e. for NNVT PMTs. Considering the absolute difference on TT between HPK and NNVT PMTs (as shown on the right of Fig. 40), as well as variations among the PMTs, the relative TT (fitted mean of the PMT hit time spectrum as previously discussed) normalized to LED_1 angle_0 ($\phi$ = 0) of each scanned PMT itself is averaged and determined for all the scanned PMTs. The relative TT non-uniformity in average of the whole photocathode of all scanned PMTs is 0.6 ns for HPK PMTs and 4.8 ns for NNVT PMTs. The NNVT PMTs show a clear dependence on the zenith angle ($\theta$) as shown on the left of Fig. 42 (relative TT increases by a factor of $\sim$ 5 between $\theta \sim$ 30$^\circ$ to $\theta \sim$ 66$^\circ$), while for the HPK PMTs the relative TT is almost constant and shows a sharp rise for $\theta$>80$^\circ$. Along the azimuthal angle, the HPK PMTs show a two-cycles-oscillation (peak-to-peak amplitude 0.6 ns) and peaks located at around $\phi \sim$ 15$^\circ$ and $\phi \sim$ 190$^\circ$), while the NNVT PMTs remain almost flat across the range of azimuthal angles.

Fig. 42

Uniformity of relative TT (normalized to the LED_0 angle_0 of each scanned PMT itself) along the zenith angle ($\theta$, left) and azimuthal angle ($\phi$, right) on average of all measurements with the same zenith angle or azimuthal angle of all the scanned PMTs. Blue: HPK PMTs; red: NNVT PMTs

4.10.5 Uniformity of PDE vs. Earth magnetic field (EMF)

Another concern of JUNO is raised regarding the correlation of the PDE uniformity versus the magnetic field strength when JUNO’s EMF shielding system has an imperfect shielding near the edges and corners of the detector, in particular for the PMTs used in the JUNO VETO system. The PDE uniformity versus magnetic field strength is further determined with the same data set as done for the PDE variation in Sect. 4.9. The sigma ($\sigma _{\textit{PDE}}$) of the 168 scanned PDEs’ distribution of a single PMT scanning is used to represent the PDE uniformity. The relative ratio of $\sigma _{\textit{PDE}}$ to that with zero-residual MF is used to represent the uniformity of PDE versus the remaining EMF. The averaged relative ratio of all the tested PMTs (9 HPK and 15 NNVT) is shown in Fig. 43. In contrast to the PDE versus MF, it seems that the uniformity of PDE of NNVT MCP-PMTs has a better tolerance than HPK dynode PMTs.

4.11 PMT long term stability

A slow aging of the PMTs used in JUNO is one of the crucial features required by JUNO for a long-term running over in a 20 to 30 years period. Long term stability measurements are of interest especially for the newly developed NNVT MCP-PMTs. Lots of studies are performed on the MCP or MCP-PMT lifetime, even with atomic layer deposition (ALD) protections [30,31,32, 98]. Concerning this issue, some R &D work had been done in [99], providing an expectation for a use over 20 years under JUNO conditions: maximum anode output charge is about 54.6 C (assuming a gain of $1 \times 10^7$, $\sim$ 50 kHz p.e.s from dark noise and signals per PMT, $\sim$ 4 Hz muon through JUNO CD LS and averaged $\sim$ 1000 p.e.s per single PMT per muon). At the Zhongshan Pan-Asia 20-inch PMT testing and potting station, the container $\#$C (as previously discussed) is re-configured to test the aging effect of HPK and NNVT PMTs under different configurations. The first loading completed its run after approx. 5 months and was intended partially for system debugging, the second loading intended for PMT long-term testing is still ongoing for more than 7 months now, where 32 PMTs (4 HPK PMTs + 28 NNVT PMTs) are separated into four groups:

• 1. Constant light to simulate 1 MHz DCR and additional 10 Hz pulses to generate 1,000 p.e.s/pulse (0.140 C/day).

• 2. 10 Hz pulses (1,000 p.e.s/pulse) to simulate accelerated aging (assuming PMT itself with 50 kHz DCR, 0.008 C/day).

• 3. Constant light to simulate 1 MHz DCR (0.138 C/day).

• 4. 1 Hz pulsed LEDs (1,000 p.e.s) to simulate the future JUNO detector (assuming 50 kHz DCR, 0.007 C/day).

Fig. 43

PDE uniformity versus magnetic field strength. Indicated data points represent means and standard deviations of the tested samples (9 HPK PMTs and 15 NNVT PMTs)

Fig. 44

Monitored DCR (left), gain (middle) and light intensity (measured number of p.e.s in µ, proportional to PDE, configured with constant light intensity, right) versus operating time under different configurations. There was an artificial shift introduced between each testing group on the plots to improve differentiation of the trend of each configuration: the first curve from top in black is group 4, the second curve from top in blue is group 3, the third curve from top in green is group 2, and the fourth curve from top in red is group 1, The y-axis is normalized to the individual starting points

Fig. 45

Working HV of potted PMTs for a gain of $1\times 10^7.$ Left: HV of bare vs. potted HPK PMTs; right: HV of bare vs. potted NNVT PMTs

Figure 44 shows the daily monitored DCR, gain, and light intensity (proportional to the PDE, configured with constant light source intensity). The DCR here is derived daily from the random coincidence of dark counts to the acquired waveform time window, which is not sensitive to instantaneous variations of the DCR. A linear fit versus the testing day is applied to each monitored curve. No obvious degradation of the DCR and measured light intensity are identified so far ($\le$ 0.0002/day, within the monitoring uncertainty), while the monitored gain of group 1 (output charge up to max. 19.6 C in total) shows a clear decreasing trend of 0.001/day (monitoring uncertainty 0.0002/day), which is as expected from the aging of the MCP plates  [30, 32, 98,99,100,101]. On the other hand, the monitored gain of other groups hints at a decreasing trend of $\sim$ 0.0002/day (monitoring uncertainty 0.0002/day), which still needs further monitoring. Even if the gain shows an obvious degradation in case of the NNVT MCP-PMTs of group 1, the requirement is still satisfied for a runtime of 20 years running ($< 50\%$ in 20 years) and deviations can be compensated by slightly increasing the working voltage of PMTs.

5 Waterproof potted PMT

All JUNO PMTs, regardless of their placement in the CD or veto, will be operated in pure water. Therefore, the JUNO PMTs must be potted (waterproof encapsulated with the PMT base firmly soldered to the PMT) before their installation [8, 18]. The potted PMTs not only need to be tested once more to guarantee the functionality after the potting process, bet also need to be crosschecked for any change in characteristics.

There are several differences between the testing configurations of bare PMTs and potted ones:

• (1) The HV divider will be soldered to PMT pins rather than using a pluggable HV divider, even though their designs are the same.

• (2) The HV-signal decoupler (specified for a positive HV) will be realized in another separate circuit, housed in a small aluminum box (as used for the functionality tests with the potted PMTs in containers #A and #B), and integrated as part of the 1F3 underwater electronics (container #D, as used in the JUNO detector) rather than being integrated into the pluggable HV divider.

• (3) Another 2 m (CD PMT) or 4 m (veto PMT) extension of the HV/signal cable (including an additional SHV connector – the bellow length of the potting structure is 0.5 m shorter than the individual cables; another 2 m extension cable is used during all the testings for easy connection) is necessary, in contrast to the bare PMTs’ testing.

The potted PMTs were tested with the same containers as the bare PMTs (containers $\#$A and $\#$B), using again the commercial data acquisition electronics (tested 1121 HPK and 3708 NNVT PMTs in total). Some of them were tested also together with the 1F3 electronics (in container $\#$D, tested 732 HPK and 1112 NNVT PMTs in total), representing the same configuration as in the JUNO detector later on [8, 12]. Since most of the investigated parameters are consistent with the results for the bare and potted measurements, only parameters only parameters which have been affected after potting will be discussed further in the following subsections.

5.1 HV of potted PMTs

After the potting, the working HV of the PMTs to apply a gain of $1\times 10^7$ is re-calibrated following the procedure presented in Sect. 4.2.2. A comparison of the new HV working point with the measured values of the bare PMTs show a good correlation (see Fig. 45). The HV increases by 30 V for HPK PMTs on average, and decreases by 30 V for NNVT PMTs on average based on the testing results from containers #A and #B. The determined HV values after potting are closer to the suggested values by the manufacturers. The variation is assumed to arise from the resistance change during production compared to the pluggable HV dividers for the HPK PMTs, and from shorter wires between HV divider and PMT pins in case of the NNVT PMTs.

5.2 Dark count rate (DCR) of potted PMTs

It was found that after the waterproof potting of the PMTs, the measured DCR was decreased by a factor of about 0.59 on average for NNVT PMTs (Fig. 46), while HPK PMTs keep their mean value stable. This effect was verified in detail for the noise level, temperature and other conditions. The mean DCR for NNVT PMTs decreases to 31 kHz tested with containers #A and #B. We conclude that this may be caused by the connection difference of the final soldered dividers and the used pluggable HV dividers, which could generate some additional noise for bare NNVT PMTs and increase the HV working point.

Fig. 46

DCR of potted PMTs at a gain of $1\times 10^7$, measured in containers $\#$A and $\#$B. Black: all PMTs; blue: HPK; and red: NNVT

5.3 Rise-time, fall-time and FWHM of potted PMTs

SPE pulses are extracted from the waveforms of each potted PMT as mentioned in Sect. 4.3 and illustrated in Fig. 10a. As known, the rise- and fall-time are more related to the PMT itself, while they also can be regulated by the HV divider design, for example, towards a slower behaviour. Figure 47 shows the rise-time, fall-time and FWHM of SPE pulses for all tested potted PMTs, measured with the container system. The average values for HPK PMTs and NNVT PMTs individually are 6.2 ns and 3.6 ns for rise-time, 9.6 ns and 15.3 ns for fall-time, and 9.6 ns and 9.9 ns for FWHM. The multi-peak structure (around 17 ns and 21 ns) of the fall-time distribution of NNVT PMTs is the joint contribution from the impedance mismatch among some of the PMTs, HV dividers, cables, SHV connectors and electronics. On the other hand, the HPK PMTs show a more uniform distribution of their waveforms among the tested PMTs.

Fig. 47

Measured rise-time (left), fall-time (middle) and FWHM (right) of potted PMTs at a gain of $1\times 10^7$ from containers #A and #B. Black: all PMTs; blue: HPK; red: NNVT

6 Summary

The performance of the JUNO 20-inch PMT system is one of the critical items to reach the designed 3% energy resolution at 1 MeV. In this article, we have described the facilities, the testing procedure and the characterization of more than 20,000 20-inch PMTs qualified for use in the JUNO detector, including both PMT types: HPK dynode PMTs and NNVT MCP-PMTs.

The testing systems, the container system and the scanning stations, have been running successfully over the whole testing campaign, achieving a satisfying noise level, measurement uncertainty, consistency between the testing systems, and monitored stability. With these systems, the absolute PDE was studied and measured based on the photon counting method, and most of the related PMT parameters were measured from the 20-inch PMTs, including the HV, gain, S/N, SPE amplitude, DCR, risetime, falltime, FWHM, charge resolution, ENF, gENF, TTS, pre-pulse ratio, after-pulse ratio, and the uniformity along the whole photocathode for various parameters such as the gain, charge resolution, PDE, TTS, and the EMF sensitivity.

Table 4 Main parameters of qualified 20-inch PMTs

All received 20-inch PMTs from HPK and NNVT have completed their qualification checks during 2017–2021. All tested PMTs have shown excellent performance, also the newly developed MCP-PMTs, and most of the PMTs reached the JUNO requirements. As a result, about 2400 PMTs (9.3%), out of the tested 22,400 PMTs have been rejected due to failing the specifications required by JUNO for visual inspection and/or electrical parameters. Except the visual defects, most of them are failures due to a low PDE (i.e. for the early version of low-QE MCP-PMTs), or due to a high DCR. The photocathode uniformity was also checked in sub-samples of PMTs with a performance around the requirements boundary, as well as for a randomly selected sample totally covering about 12% of all HPK PMTs and about 18% of all NNVT PMTs. Finally, a total of 20,062 PMTs were successfully qualified for use in JUNO (typical parameters shown in Table 4), with a mean PDE of 28.5% for HPK dynode PMTs and 30.1% for NNVT MCP-PMTs (low-QE PMTs 27.3%, and high-QE PMTs 31.3%), and the expected mean PDE of the JUNO CD (17,612 PMTs, the PMTs will be randomly distributed by type (Dynode / MCP) as well as by performance in the CD.) and veto detector (2400 PMTs) is 30.1% and 26.1% with the proposed schema, respectively. All the measured parameters of the qualified PMTs are stored in the JUNO PMT database, so that they can be accessed and used by the collaboration for detector simulations, during installation and commissioning, and eventually for the data analysis.

Furthermore, about 5000 of the waterproof potted PMTs (encapsulated with the PMT base firmly soldered to the PMT) are tested again after the potting in the testing facilities. Additionally, dedicated tests including JUNO final electronics have been performed as well to study the long-term behaviour of the JUNO PMTs under normal conditions and higher strain to simulate and investigate possible aging effects.