numRecipes

Class PoissonBinnedLikelihoodCalculator

java.lang.Object

numRecipes.PoissonBinnedLikelihoodCalculator

All Implemented Interfaces:

java.io.Serializable

public class PoissonBinnedLikelihoodCalculator
extends java.lang.Object
implements java.io.Serializable

Calculate the binned Poisson Likelihood and its p-value for a series of (N_i, \mu_i) where N_i is the observed number and \mu_i is the predicted number at the i-th bin. You should call the method fillData(in) to make map of (N_i, \mu_i) Written originally by S. Yoshida for the IceCube EHE analysis. 2012/8/15

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
boolean	debugFlag
static double	default_precision
protected double	poissonLogLikelihood
protected java.util.Map	realDataMap Map container for (N_i, \mu_i).
protected java.util.Map	replicaDataMap
protected boolean	useTheReplicaResults

Constructor Summary

Constructors

Constructor and Description
PoissonBinnedLikelihoodCalculator() Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
void	addExpectedNumbers(double signalFactor, PoissonBinnedLikelihoodCalculator calSignal) Add the expected number in calSignal to the internal map to this object.
void	addExpectedNumbers(PoissonBinnedLikelihoodCalculator calSignal) Add the expected number in calSignal to the internal map to this object.
void	copyExpectedNumbers(double signalFactor, PoissonBinnedLikelihoodCalculator calSignal) Copy the expected number in calSignal to the internal map to this object.
void	copyExpectedNumbers(PoissonBinnedLikelihoodCalculator calSignal) Copy the expected number in calSignal to the internal map to this object.
void	copyObservedNumbers(PoissonBinnedLikelihoodCalculator calSignal) Copy the observed number in calSignal to the internal map to this object.
void	copyPoissonBinnedData(PoissonBinnedLikelihoodCalculator cal) copy the Poisson bin data (observed number, expected number) stored in PoissonBinnedLikelihoodCalculator cal.
void	copyPoissonBinnedDataMap(java.util.Map poissonDataMap) copy the Map containing the Poisson binned data.
void	copyTheReplicaResults() This method copies the map to store the results frmo the replica experiment to the map for storing the real results.
void	fillData() Reading a series of (N_i, \mu_i) from the standard input by the interactive methof
void	fillData(java.io.DataInputStream in) Reading a series of (N_i, \mu_i) from the input stream.
double	getLikelihood() Calculate the likelihood to the series of (N_i, mu_i).
double	getLikelihood(double scaleFactor) Calculate the likelihood to the series of (N_i, mu_i*scale_factor).
double	getLikelihood(double scaleFactor, double signalFactor, PoissonBinnedLikelihoodCalculator calSignal) Calculate the likelihood to the series of (N_i, bg_mu_i+sig_mu_i*signalFactor).
static double	getLogPoissonProbability(long nObserved, double expectedValue) Calculate -log(Poisson) for a given set of (nObserved,expectedValue).
double	getPvalue() Calculate the p-value of the experiment to yield the series of bins (N_obs, \mu_i) with the default_precision.
double	getPvalue(double precision, double scaleFactor) Calculate the p-value of the experiment to yield the series of bins (N_obs, \mu_i).
double	getSignalFactor(double scaleFactor, PoissonBinnedLikelihoodCalculator calSignal) Calculate the signalFactor to maximize likelihood to the series of (N_i, bg_mu_i+sig_mu_i*signalFactor).
double	getSumOfExpectedValues() Return the sum of the expected values \mu_i.
long	getSumOfObservedValues() Return the sum of the observed numbers \N_i.
static PoissonBinnedLikelihoodCalculator	inputPoissonBinnedLikelihoodCalculator(java.io.InputStream in)
static void	main(java.lang.String[] args) A simple main method
static void	outputPoissonBinnedLikelihoodCalculator(PoissonBinnedLikelihoodCalculator cal, java.io.OutputStream out)
void	printLogLikelihood()
void	printLogLikelihood(boolean printData) print out the calculated Poisson binned log likelihood to the standard output
void	replaceExpectedNumbersWithObservedNumbers() Replace the expected number with the observed number so that the expected number vector exactly equals to the observed number vector.
void	replaceExpectedNumbersWithObservedNumbers(PoissonBinnedLikelihoodCalculator calBase) Replace the expected number with the observed number - expected number in calBase so that the expected number vector maximizes the likelihood under the conditaion that the minimum expected numbers are bounded by the expected numbers stored in calBase.
protected void	runPoissonExperiment() This method is purely for debugging purposes.
double	runReplicaExperiment() Run replica expetiment one time following the Poissonian to the series of (N_i, mu_i).
double	runReplicaExperiment(double scaleFactor) Run replica expetiment one time following the Poissonian to the series of (N_i, mu_i).
void	useTheResultsByTheRealExperiment() call this method before calling the getLikelihood method if you want to calculate the log likelihood by the real results read by the method fillData().
void	useTheResultsByTheReplicaExperiment() call this method before calling the getLikelihood method if you want to calculate the log likelihood by the results by the replica experiment you had run withe the method runReplicaExperiment()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

useTheReplicaResults

protected boolean useTheReplicaResults

default_precision

public static final double default_precision

See Also:

Constant Field Values

realDataMap

protected java.util.Map realDataMap

Map container for (N_i, \mu_i). N_i is the observed number in the real experiment

replicaDataMap

protected java.util.Map replicaDataMap

poissonLogLikelihood

protected double poissonLogLikelihood

debugFlag

public boolean debugFlag

Constructor Detail

PoissonBinnedLikelihoodCalculator

public PoissonBinnedLikelihoodCalculator()

Constructor.

Method Detail

getLogPoissonProbability

public static double getLogPoissonProbability(long nObserved,
                              double expectedValue)

Calculate -log(Poisson) for a given set of (nObserved,expectedValue).

fillData

public void fillData(java.io.DataInputStream in)
              throws java.io.IOException

Reading a series of (N_i, \mu_i) from the input stream. The format is

       nObserved_0 Expected-value_0 
       nObserved_1 Expected-value_1 
       .....
       

You need a "space" before each end of line.

Throws:

java.io.IOException

fillData

public void fillData()
              throws java.io.IOException

Reading a series of (N_i, \mu_i) from the standard input by the interactive methof

Throws:

java.io.IOException

printLogLikelihood

public void printLogLikelihood(boolean printData)

print out the calculated Poisson binned log likelihood to the standard output

printLogLikelihood

public void printLogLikelihood()

getSumOfExpectedValues

public double getSumOfExpectedValues()

Return the sum of the expected values \mu_i.

getSumOfObservedValues

public long getSumOfObservedValues()

Return the sum of the observed numbers \N_i.

copyPoissonBinnedData

public void copyPoissonBinnedData(PoissonBinnedLikelihoodCalculator cal)

copy the Poisson bin data (observed number, expected number) stored in PoissonBinnedLikelihoodCalculator cal.

copyPoissonBinnedDataMap

public void copyPoissonBinnedDataMap(java.util.Map poissonDataMap)

copy the Map containing the Poisson binned data. You can call this method and put the data map generated by any external object so that this PoissonBinnedLikelihoodCalculator object gets functioning without calling the method fillData(). The map structure should be:

      Map((Integer)BinNumber, LinkedHashMap((Long)Observed Number, (Double)Expected Number))
      

runReplicaExperiment

public double runReplicaExperiment(double scaleFactor)

Run replica expetiment one time following the Poissonian to the series of (N_i, mu_i). Return log likelihood -log(PoissonLikelihood). You must fill the data of (N_i, mu_i) first by, for example, calling the method fillData(DataInputStream in).

       double scaleFactor:  scaleFactor which is multiplied to mu_i; if set to 1.0, no fudge factor is introduced.
       

runReplicaExperiment

public double runReplicaExperiment()

Run replica expetiment one time following the Poissonian to the series of (N_i, mu_i). Return log likelihood -log(PoissonLikelihood). No fudge factor, i.e., scaleFactor=1.0.

useTheResultsByTheReplicaExperiment

public void useTheResultsByTheReplicaExperiment()

call this method before calling the getLikelihood method if you want to calculate the log likelihood by the results by the replica experiment you had run withe the method runReplicaExperiment()

useTheResultsByTheRealExperiment

public void useTheResultsByTheRealExperiment()

call this method before calling the getLikelihood method if you want to calculate the log likelihood by the real results read by the method fillData(). This is the default setting.

copyTheReplicaResults

public void copyTheReplicaResults()

This method copies the map to store the results frmo the replica experiment to the map for storing the real results. Mostly for debugging purposes, but you may need to call this method to study the likelihood ratio distribution with the alternative (signal+bg) hypothesis.

getLikelihood

public double getLikelihood(double scaleFactor)

Calculate the likelihood to the series of (N_i, mu_i*scale_factor). Return log likelihood -log(PoissonLikelihood). You must fill the data of (N_i, mu_i) first by, for example, calling the method fillData(DataInputStream in).

       double scaleFactor:  scaleFactor which is multiplied to mu_i; if set to 1.0, no fudge factor is introduced.
       

getLikelihood

public double getLikelihood(double scaleFactor,
                   double signalFactor,
                   PoissonBinnedLikelihoodCalculator calSignal)

Calculate the likelihood to the series of (N_i, bg_mu_i+sig_mu_i*signalFactor). Return log likelihood -log(PoissonLikelihood). You must fill the data of (N_i, mu_i) first by, for example, calling the method fillData(DataInputStream in). PoissonBinnedLikelihoodCalculator calSignal stores the binned poisson distribution for the signal hypothesis.

       double scaleFactor:  scaleFactor which is multiplied to bg_mu_i; if set to 1.0, no fudge factor is introduced.
       

getSignalFactor

public double getSignalFactor(double scaleFactor,
                     PoissonBinnedLikelihoodCalculator calSignal)

Calculate the signalFactor to maximize likelihood to the series of (N_i, bg_mu_i+sig_mu_i*signalFactor). You must fill the data of (N_i, mu_i) first by, for example, calling the method fillData(DataInputStream in). PoissonBinnedLikelihoodCalculator calSignal stores the binned poisson distribution for the signal hypothesis.

       double scaleFactor:  scaleFactor which is multiplied to bg_mu_i; if set to 1.0, no fudge factor is introduced.
       

getLikelihood

public double getLikelihood()

Calculate the likelihood to the series of (N_i, mu_i). No fudge factor.

copyObservedNumbers

public void copyObservedNumbers(PoissonBinnedLikelihoodCalculator calSignal)

Copy the observed number in calSignal to the internal map to this object.

copyExpectedNumbers

public void copyExpectedNumbers(double signalFactor,
                       PoissonBinnedLikelihoodCalculator calSignal)

Copy the expected number in calSignal to the internal map to this object. This method is used when the null pypothesis is NOT background only but signalFactor*signal + background.

copyExpectedNumbers

public void copyExpectedNumbers(PoissonBinnedLikelihoodCalculator calSignal)

Copy the expected number in calSignal to the internal map to this object. This method is used when the null pypothesis is NOT background only but signal + background.

addExpectedNumbers

public void addExpectedNumbers(double signalFactor,
                      PoissonBinnedLikelihoodCalculator calSignal)

Add the expected number in calSignal to the internal map to this object. This method is used when the null pypothesis is NOT background only but signalFactor*signal + background.

addExpectedNumbers

public void addExpectedNumbers(PoissonBinnedLikelihoodCalculator calSignal)

Add the expected number in calSignal to the internal map to this object. This method is used when the null pypothesis is NOT background only but signal + background.

replaceExpectedNumbersWithObservedNumbers

public void replaceExpectedNumbersWithObservedNumbers()

Replace the expected number with the observed number so that the expected number vector exactly equals to the observed number vector. This operation is used for building a saturated likelihood.

replaceExpectedNumbersWithObservedNumbers

public void replaceExpectedNumbersWithObservedNumbers(PoissonBinnedLikelihoodCalculator calBase)

Replace the expected number with the observed number - expected number in calBase so that the expected number vector maximizes the likelihood under the conditaion that the minimum expected numbers are bounded by the expected numbers stored in calBase. calBase usually stored the numbers expected in the pure background hypothesis. This operation is used for building a saturated likelihood.

getPvalue

public double getPvalue(double precision,
               double scaleFactor)

Calculate the p-value of the experiment to yield the series of bins (N_obs, \mu_i). You must fill the data of (N_i, mu_i) first by, for example, calling the method fillData(DataInputStream in). The MC method with the binned Poisson likelihood computes the p-value.

       double precision :  precision you want to caluclate p-value. Running replica expetiments 1/precision times
                           to compute p-value.
       

getPvalue

public double getPvalue()

Calculate the p-value of the experiment to yield the series of bins (N_obs, \mu_i) with the default_precision.

runPoissonExperiment

protected void runPoissonExperiment()

This method is purely for debugging purposes. Run an expetiment to yield N_i from the expected value \mu_i following Poissonian. Then you can check if the calculated p-value makes sense by calling getPvalue(). You must fill the data of (N_i, mu_i) first by, for example, calling the method fillData(DataInputStream in).

outputPoissonBinnedLikelihoodCalculator

public static void outputPoissonBinnedLikelihoodCalculator(PoissonBinnedLikelihoodCalculator cal,
                                           java.io.OutputStream out)
                                                    throws java.io.IOException

Throws:

java.io.IOException

inputPoissonBinnedLikelihoodCalculator

public static PoissonBinnedLikelihoodCalculator inputPoissonBinnedLikelihoodCalculator(java.io.InputStream in)
                                                                                throws java.io.IOException

Throws:

java.io.IOException

main

public static void main(java.lang.String[] args)
                 throws java.io.IOException

A simple main method

Throws:

java.io.IOException