28.06.17

Transverse Variables:

I've now got the spectra for each of the new kinematic variables: TransverseVariableSpectra.pdf

Looking into:

-Why are there a gaps in the peaks in 2D spectra

-Are the events which are all in extreme bins exactly cos = +/-1

-Normalising bins by area

-Are these physically what we expect e.g why does 0piNp favour cos alpha and cos phi =-1

I've looked at how changing each parameter of the cross section model by +/- 1 sigma affects these spectra.

To avoid showing ~5500 plots, I've just attached results for varying the MAQE parameter by -1sigma: dP.pdf, dalpha.pdf, dphi.pdf

Seems to be less of an effect when there's a pion involved (but note the z axis scales change)

CCD Characterisation:

With Yuri's help, have got Gabriela's ccd characterisation code running for analysis of dark box runs. However, the first plot I've got out for 1x1 binning shows us getting less noise than predicted using the manufacturer's specifications.

noise_exposure_1x1.gif

I'm currently checking through the code how the predicted noise is calculated. I've emailed Gabriela about the rms value I've used for the measured noise, and also what the error bars mean. I've just left them in from what was in the code for now. I will also check with her about how she scaled with temperature as that might be where differences have entered.

Note: The 10s data point here is at a different temperature to the others so it is not surprising that it doesn't rise as much as the prediction. I will change this so the prediction accounts for this too.

2x2 binning:

noise_exposure_2x2.gif

21.06.17

Transverse Variables:

I ran a fit in p-cos theta with the events scaled by 0.1. The results from this are in agreement with the same fit but with the version of MaCh3 that doesn't take in transverse variables as well.

XsecComparison.gif

FluxComparison.gif

These have Clarence and Patrick's 'blessing' that the results agree and differences are due to statistical fluctuations. The consistently higher flux parameters for 'with transverse' is likely due to high dimensional marginalisation effects.

So now have started running fit using one of the new variables: dP-cos theta.

First I rebinned the fake data in delta P:

0pi1p.gif0piNp.gif

1pi1p.gif1piNp.gif

The fit is still running at the moment.

Resolution Scan:

I've had multiple crashes while running fits scanning through different resolutions. These started when I started running fits with the transverse variables at the same time. When I'd checked out a new branch of MaCh3 for implementing the transverse variables I'd got some paths pointing to the same places in each so I think that's what the problem was. So I'm now continuing to run fits at different momentum/angle resolutions.

I've also run a fit for ND280 energy thresholds to compare to HPTPC. The constraints are better for most parameters but not all. The parameters with the lowest ratios are the ones I will focus on for presenting the resolution scan.

HPTPC_ND280.gif

NB: For these fits MEC parameters are still ~0.27 to be consistent with the rest of the momentum scan I'm doing. Subsequent fits will have them set to 1.

CCD Characterisation:

Have taken lots of data in the dark box using the 4 shooter. These are at different exposure length and binning to be able to reproduce plots in Gabriella's thesis for comparison.

There was an issue with there being a limit on the number of images taken which Yuri solved.

Since the hot weather started, the camera could not get to -25 degrees, so data was taken at a warmer temperature so I'll have to scale it when comparing to manufacturer's specifications. This was done for the DMTPC but in the other direction.

I've been having trouble running the code for the noise calculation, but this does the same job as our 'my_first.cpp' so could proceed with that. The code for comparing to manufacturer spec seems to run fine.

Now have first images taken on vessel! These are on the DAQ at the moment as there's been issues getting it synced/copied now there's a large amount of data on there, and don't want to be running analysis on there.

14.06.17

Resolution Scan:

Scan with events scaled by 0.1 is currently running.

I will plot the ratio of each xsec parameter value uncertainty for HPTPC to ND280, and will then focus on a few parameters where HPTPC has the greatest effect.

For these, I'll plot the uncertainties at different resolutions.

But haven't done an ND280 fit with events scaled by 0.1. This is now running.

In the mean time, below is plot of ratios of uncertainties to show what I mean. This is the ratio of 2x the resolution to original HPTPC fit. i.e for values <1, having better resolution has decreased the uncertainty:

pLow_Ratio.gif

The values seem quite scattered, so I will look at larger differences in resolution as well.

I also am looking into limits on resolution from the magnets and ultimate theoretical resolutions: fermi momentum for momentum, pixel size for angle.

Transverse Variables:

I've coded the modifications to MaCh3 to accommodate the new transverse variables (dAlpha, dPhi,dP).

TransverseVariables.png

I've used this run a fit, but with the old variables (cos theta and p) to check it still runs as expected. When I plot the ratios of uncertainties for this fit vs the original p-costheta fit they've changed quite a lot:

RatioPlot.gif

However, the original fit used in this plot was one of the first I got out and I've changed parts of samplePDF since then. I'm now running a fit on the transverse variables version of MaCh3 (but with p costheta) with the events scaled by 0.1, so I can compare to an up to date fit.

Once I'm sure the changes haven't affected anything they shouldn't, I will run a fit in the new variables. The goal is to see which combination of variables constrains the uncertainties the most.

4 Shooter Characterisation:

Initially took a 50 frame run to see how temperature of camera increasing with each image taken affects the counts in each pixel (error bars left off as they are quite big due to the spot in the image):

Screen_Shot_2017-06-14_at_13.42.06.png

Want to see if this increased indefinitely or levelled off, so I've taken a long (100 frame) run on in the dark box. The mean counts per pixel for each image does not seem to increase though. The only change is there's now a block in the dark box, which gets rid of the spot in the images.

Graph1.gif


my_first_hsub_r01164033_e099_c0.png

I now have code from Gabriella for characterising the camera noise and comparing with the company specifications and converting to physical units of electrons. Also got her thesis chapter on this to guide me.

When we know camera is working as it should we can light leak check, and want to be doing this for each flange installation.

Once 4 shooter running will do same characterisation for 1sh using windows interface. Will then try to help Yuri integrate 1sh into linux

07.06.17

Mark Scott told me how to correct the bug that was causing the cut off in HPTPC fake data. This was due to the order in when we were checking if the highest momentum negative track exists and selecting the lepton track. This solves the issue, as an example, the momentum-theta plot for CC0pi0p events is below:

cc0pi0p.gif

Currently running a full fit to check this hasn't affected results.

I've got first results from a resolution scan for the HPTPC. I changed the resolution by a factor of 2 for both the angle and momentum and all combinations of each. The original resolution was 1-13% for momentum, 0.004 for cos theta. The resulting cross section plots are below, but I need to look into better ways of presenting this, maybe having the original resolution values on the same plot for comparison. When I've analysed these results properly I will look at what best to change next i.e going up or down by higher amounts for either momentum or angle. Here higher/lower refers to smearing, so lower is better resolution.

Higher Momentum, Original Theta

HighP_OrigT.gif

Original Momentum, Original Theta

xsecNoHMNT.gif

Lower Momentum, Original Theta

LowP_OrigT.gif

Higher Mom, Lower Theta

HighP_LowT.gif

Original Mom, Lower Theta

OrigP_LowT.gif

Lower Mom, Lower Theta

LowP_LowT.gif

Higher Mom, Higher Theta

HighP_HighT.gif

Original Mom, Higher Theta

OrigP_HighT.gif

Lower Mom, Higher Theta

LowP_HighT.gif

Last Thursday I went to see Patrick and we got started on implementing transverse variables into MaCh3 fits. This is involving changes to samplePDFND2014 which I'm still doing but know what I need to do in the immediate future. I'll now go every Thursday.

I haven't had chance to finish off the macro to look at total noise from dark box images to see if the camera is heating up with each image taken, but is ongoing.

31.05.17

Went to Collaboration Meeting in Tokai, and gave update at ND Upgrade workshop.

Here, I spoke to Mark Scott about the cutoff in the fake data at around cos theta = 0.2. I was able to show him exactly what was happening, and he said there was likely a bug. Have left it with him for now.

Also spoke to Simon Bienstock about new splines he'd sent me. These are compatible with new cross section. I've got latest version of MaCh3 also compatible with new cross section, but getting crashes. Simon said the splines files may well not work with MaCh3. I think this is the case rather than it being an issue with new version of MaCh3.

Still running resolution scan for HPTPC momentum and angle. Haven't got on a cluster computer yet, will do this asap.

Going to Imperial tomorrow to talk about getting transverse variables involved in these fits. Will now go every Thursday so should start making progress with this now.


Yuri's shown me how to use m3DAQ for taking dark box images, and software for analysis. I'm writing a macro to get integrated noise for each image in a run. Want to see if this increases with number of images taken, to see if camera is warming and causing an effect. Here's an example of an image taken with 2 second exposure, showing large 'spot':

r1150013.png

In the lab: Leak checking and vessel conditioning are still dependent on parts. TPC construction starting when clean tent clean, or construct then clean downstairs?

Particle Counts:

Location Particle Size (micro meters) Count
Outside 0.3 40,274,325
Outside 0.5 5,941,702
Outside 5.0 105,591
     
Outside 0.3 39,234,569
Outside 0.5 5,025,639
Outside 5.0 44,143
     
Outside 0.3 41,578,851
Outside 0.5 5,472,723
Outside 5.0 41,671
     
Inside 0.3 39,694,457
Inside 0.5 6,070,955
Inside 5.0 92,171
     
Inside 0.3 39,730,478
Inside 0.5 5,950,531
Inside 5.0 102,413
     
Inside 0.3 41,019,113
Inside 0.5 5,570,191
Inside 5.0 32,843
     
Outside Average 0.3 40,362,582
Outside Average 0.5 3,669,670
Outside Average 5.0 163,624
     
Inside Average 0.3 40,148,016
Inside Average 0.5 5,863,892
Inside Average 5.0 75,809


Last update I showed results of a fit where all events were scaled by 0.1. For the MEC (O) parameter of cross section model, the pre and post fit values differed. This is due to the gaussian distribution being cut off at 0:

MEC_O.gif

17.05.17

Am proceeding without the backward going HMNT as this gets rid of cut off at cos theta = 0.2.

I've rebinned the fake data so the bigger bins at higher momenta aren't filled as much. This is shown below for CC0pi0p events:

CC0pi0p_NoBkwdHMNT.gif

I fitted this fake data, running a 500,000 step chain:

xsecNoHMNT.gifFluxNoHMNT.gif

I've included my previous results to show the differences:

Xsec.gifFlux.gif

Improvement in match up of pre and post fit values for NC Coherent Norm. But worse for Eb (O). Still get 'over-constraint' for CA5 RES, MA RES, Bg RES, and CC Nue Norm.

The flux parameters are much the same.

Scaling Events:

I then ran a chain where each event is scaled by a factor of 0.1. This was because I hadn't been worrying about POT weighting while just trying to get a fit out that worked.

xsec_scale01.gifflux_scale01.gif

Now MEC(O) differs from its prior value. This makes me think it's down to low stats as the values jump around quite a lot but I need to look into this a bit more.

Now get less 'over-constraint' on CA5 RES, MA RES, and Bg RES, but still there for CC Nue Norm.

To compare effects of scaling events, below are the posterior plots for the first 4 flux parameters, as well as MAQE, MA RES, and NC Coherent Norm.

scale_noScaleOverlay_b0.gifscale_noScaleOverlay_b1.gifscale_noScaleOverlay_b2.gifscale_noScaleOverlay_b3.gifscale_noScaleOverlay_MAQE.gifscale_noScaleOverlay_MARES.gifscale_noScaleOverlay_NCCoherentNorm.gif(Note the x axis scale change for NC Coherent Norm)

Wider gaussians so larger uncertainty for scaled events. MAQE and MARES seemed to be over constrained, but still have gaussian shape. For NC Coherent Norm, can see peak has moved slightly.

Am going to look more into what each parameter is to try to understand why some of the values change between these fits, or if its just stats.


HPTPC Resolution Scan:

I've started a chain where I've changed the momentum resolution of the detector. I'm going to scan through different resolutions for momentum and angle to see effect on constraints.

Chain running at the moment has increased momentum resolution by a factor of 2, but is still running. Will need to run lots of chains to complete this study but each is taking ~1.5 days. I spoke to MaCh3 people about changing step size but already at roughly optimal fraction of events accepted (step size 0.05, ~20% accepted). Perhaps could use less steps (running 500,00 at the moment) for this study? Or just keep going.


Latest Cross Section Model:

Have installed latest version of MaCh3 and made my changes so it is now compatible with proton multiplicity samples and psyche truth selections.

It all compiles now, I just need to update where the splines files are and then can try to run a chain.

I don't expect the chain to run first time out the box but on course to have it running by the end of the week.


Lab:

M3 Slow and DAQ hard drives are now in their original boxes from MIT. These have new IP addresses:

  • m3daq.pp.rhul.ac.uk 134.219.109.68
  • m3slow.pp.rhul.ac.uk 134.219.109.69
The old boxes, cypress and lab-dm-011 retain their original IPs.

Both ethernet sockets (T/061 & T/062) in the TPC lab are connected to the PP network. Ethernet cable had been cut in wall but that is now fixed.

Waiting on orders before leak checking and vessel conditioning can take place. Will be early next week.

We've started on TPC stand, then can start TPC construction.


Also, I think my abstract has been accepted for PGR conference so need to prepare that and nearer the time request a practice talk. And need to finish 1st year report.

10.05.17

Now have full splines files:

  • One for each configuration of ND280 upgrade
  • Some of these are compatible with latest xsec model
Now getting up to date version of MaCh3 to be able to use this up to date xsec model

And making the changes I'd made to my version to this newer version so that it will be compatible with the psyche truth selections and proton multiplicty samples I've been using.

I've also been looking at why there's a cut-off in the HPTPC fake data at cos\theta = 0.2:

numuCC0pi0p_1.gif

I started truth smearing events from the Genie T2K flux to see if there was anything similar.

Didn't see this effect, but plot is for all CC events (the original cut-off was seen in all samples, but is strongest in cc0pi0p)

GenieHPTPC.gif

See quite a different spectrum for a few reasons. These events were on argon and using different selection efficiencies and detection thresholds. Also using different binning.

I didn't finish trying to recreate the effect in Genie as have been looking into the psyche selection truth code.

We have a backward going selection efficiency and a forward going selection efficiency, with the change between the two being at cos\theta = 0.2.

At the moment both these efficiencies are the same so in theory this shouldn't cause the issue.

But when the cross-over point is changed to 0.3, the cutoff in the spectrum moves to here:

MovedCut.gif

So this is clearly where the effect is being introduced.

I spoke to Mark Scott about this, and he saw a bug in the code which would cause this sort cut off if the two efficiencies were different. I made the corrections but still saw the effect.

To further check this is where issue is I commented out one of the parts where backward efficiencies were used. This gives a more reasonable spectrum (below).

However, this is not correct, we're missing out on backward going leptons. This is just to show this is where the error comes in.

NoCut.gif

Part of issue is I don't fully understand the code here. But as there was bug before hopefully there is just another one and it can be easily fixed, rather than an intrinsic problem with the geometries used in the simulation.


For the TREx-MaCh3 interface, I've emailed Jen a list of the variables needed for the inputs.

She's going to send the formats of how these are outputted.

Once I've got those and have started coding up the interface properly it should become clearer exactly what's required and can then get more information from Jen if needed.

04.05.17

Since last meeting I rebinned the HPTPC fake data being produced in MaCh3. The old binning was designed for 2\pi angular coverage, but have 4\pi for HPTPC. Below is a comparison of old and new binning for cc0pi0p sample:

cc0pi1p.gifnumuCC0pi0p_1.gif

There seems to be an unexpected cut-off at cos\theta = 0.2. Taking the projection to the cos\theta axis shows this more clearly:

numuCC0pi0p_projY.gif

The cut-off seems to be in all samples but is most prominent in cc0pi0p.

I've emailed Mark Scott these plots to see if he knows what is going on.

Am also in process of applying truth smearing to the Genie T2K flux file to see if there's anything similar.


I've now got the first nominal fit working. It was 500,000 steps. The comparison plots of posterior and prior parameter values and uncertainties are shown below for the cross section and beam models:

Xsec.gif Flux.gif

This is pretty much as expected. The prefit and postfit values match each other closely, and the uncertainties have been constrained for all parameters.

However, in the cross section model, CA5 RES and MaRES are constrained by a lot. Probably more than would be expected.

Another unrelated concern was that one error I'd earlier run into was flux weightings being set to -999. When this happened I just set them to 1 and never got to the bottom of why it happened.


Also in the process of getting larger splines files. These have been produced by Simon Bienstock who has put them on IRODs for me and I am in the process of getting access/permissions to download these. Some of them are compatible with the latest xsec model.

Then next steps are adding anti neutrino samples, including proton and pion information and transverse variables, and using the updated xsec model when it's merged.

12.04.17

Think I've almost got the fit behaving as expected.

I hadn't been adding the fake data in my MaCh3 executable in the right way to be fitted.

I ran a shorter (50,000) step chain and the mean values of the parameters seem to match the inputted values.

A 500,000 step chain died last night (due to linappserv rather than code crashing), but got ~130,000 steps.

Log Likelihood convergences:

130LogL.gif130Flux.gif130Xsec.gif

Flux parameters:

130b_0.gif130b_10.gif130b50.gif

Xsec parameters:

130Xsec0.gif130Xsec10.gif130Xsec15.gif

Table.jpeg

Cross-section parameters all closely follow priors now.

Half a million step chain is currently running.

05.04.17

Found issue with xsection weightings. These were also being read from splines file in a way I hadn't realised.

When this was set to read in as CC0pi0p for all events the flux weighting was more reasonable, and samplepdf integrals are no longer negative or zero.

I ran a 500,000 step chain overnight, and LogL converges:

LogL.gifLogLsmall.gif

LogLsample.gif

But some of the parameters still don't look right.

xsec0.gifxsec20.gif

b10.gifb_20.gif

Haven't had chance to fully investigate if widths have been reduced, and why flux parameters are >>1.

Could be as just setting flux weight to 1 when it was <0. Never got to bottom of why or where it was being set to -999 for some events.

Ongoing:

  • TREx - MaCh3 interface, waiting for output file from Jen/Paula at Warwick, and will go to Imperial once a week to work on this with Patrick
  • Proton and Pion Momentum and angle, and using transverse variables. Want initial fit out before implementing this
Also been preparing talk for T2KUK F2F and helping in lab

29.03.17

By setting flux weighting to 1 whenever it was <0 (had been -999 for certain events), have got rid of the 'curtain' of points above the convergence line for the sample log likelihood (before on the left, after on the right):

LogL_sample.gifFit8LogL_sample.gif

Total log likelihood still converges:

Fit8LogL.gif

However, for individual parameters of flux and cross-section, have quite un-gaussian shapes:

fit8b_0.giffit8b_10.giffit8b_20.gif

fit8xsec_0.giffit8xsec_10.giffit8xsec_20.gif

When I was looking at the flux reweighting, I never tracked down exactly where the weights were being changed, and the integral of the samplepdfs are still 0 for some of the samples.


On Friday, I spoke to Patrick about getting pion and proton information in my MaCh3 fits.

What I've done has given me crashes, but should be fairly straight forward so just need to spend some time looking into that. I've commented it out for now while I'm getting original fit to work.

Also discussed smoothing the interface between TREX and MaCh3, so that information from TREX outputs can be inputted into MaCh3.


Also have been helping unpack equipment in lab, and preparing talks for symposium and T2K UK meeting.

22.03.17

Working through issues with truth studies for HPTPC with MaCh3:

  • Last week had error message as the truth vertices from the splines file and psyche weren't matching up
  • This was because, like the arrays in the splines file I'd had problems with in the previous week, there wasn't enough elements for the samples I was using
  • This is also the case for all the other arrays in the splines file
  • So I've added an if statement so that whenever I'm going into these arrays, the event is treated as being CC0pi0p
  • With this was able to run a 50,000 step chain for a near detector fit
  • But had two error messages
- One event in ~530,000 still didn't have the truth vertices matching

- One event wasn't being found at all in the splines file

  • Was still able to complete the fit though
  • LogL converges after initial burn-in phase, as shown below
LogL.gif
  • But for sample Log Likelihood there's a lot of deviation from this convergence
LogL_sample.gif
  • Unlikely this is caused by the above errors as they were just for two individual events
  • But then noticed that the integral of the samplepdfs were incorrect (0 or large negative)
  • So looked into reweight function, and the fluxweighting is occassionally -999 (rest are near unity)
  • This is from when event is loaded, rather than any of the reweighting multipliers (from splines, POT weights, or another float 'weights')

15.03.17

Still trying to fit HPTPC fake data with MaCh3.

  • Had issues with the detector systematics covariance file. There's a TObjArray within the file with an axis for each of the original samples. I now have more samples than this (extra: CC0pi0p, CC0pi1p, CC0piNp, CC1pi0p, CC1pi1p, CC1piNp).
  • But this was for ND280 anyway as don't have HPTPC detector covariances yet, so am now ignoring detector systematics.
  • Then had troubles with cross-section systematics. In the psyche `makefakedata` equivalent, all events are assumed to be 0pi0p and there is only information for this sample.
  • Emailed Mark Scott, from whom I got this version of psyche, and he confirmed that's what he'd done.
  • So have inputted similar case into my version of MaCh3 for now.
  • Now getting error as 'Chain vertex' and 'psyche vertex' are not equal.
  • I think it's an issue with getting truth vertex from splines.
  • Currently trying to get to the bottom of this.
Have presented an example of the fake data below, just to show that with all the changes/errors this has not been affected. This is for the momentum and angle of the outgoing lepton in CC0pi0p interactions (for continuity with previous weeks).

HPTPC_numuCC_0pi0p_5000.gif


Also battling blue screen of death laptop problems at the moment.

08.03.17

MaCh3 execuable now adds the correct Truth Selections. Am able to recreate the fake data histograms I produced in psyche with makefakedata.exe.

Below is an example of this, for the momentum and angle of the outgoing lepton in CC0Pi0P interactions.

HPTPC_numuCC_0pi0p_5000.gif

So I then went about fitting this with MaCh3.

After a few issues with crashes, it ran and I produced a 500,000 step chain.

The likelihood is shown below:

LogL.gif

Burn in phase ~10,000 steps.

However, the uncertainties on the parameters of the cross section, detector, and flux models are not constrained. Most seem to be within 5-10% of initial values (root of the diagonal elements of the covariance matrices).

For example, nddet_cov[350][350] = 0.003467, so the uncertainty is (0.003467)^(1/2) = 0.05888

But RMS of posterior is 0.06121:

ndd_350.gif

Getting some code from Clarence at Imperial to try and diagnose what's going wrong.

Have also run a 5,000,000 step chain but haven't fully looked at the results yet, but the few I have looked at don't show much improvement.


Last week had problems as some of the Truth Selections required a file I didn't have. This was ND280 upgrade specific so I didn't need it at the time but will be doing ND280 upgrade studies soon so have now got this code. It contains 2D histograms (cos\theta and momentum) of the efficiencies for muons, protons, and pions, reconstruction, and mis-identification of particles as muons and pions.

01.03.17

Have now produced a MaCh3 executable which adds the correct Truth Selections
Initially had issues with the syntax of the name of selections
Then selection source codes were looking for a file which I didn't have, but this file isn't needed in HPTPC truth selections

  • Emailed Simon (whose directory the file is in), he could send it but it's ND280 upgrade specific and also the file needed depends on which version of truth selections I'm using
  • The file is efficiencies.root, containing efficiencies for muons, pions, and protons, and misId for each of the particles wanted to be reconstructed as muon or pions
Then had break segementation violation as when filling the data from the samples it was looking for Momentum rather than TrueMomentum.
  • I've hard coded an if statement to change between Truth and Reconstructed lepton candidates.
Now getting errors going over edge of an array when filling bins, so currently trying to see if that's due to something I've changed or something I should change
Once finished debugging, will have fake data for HPTPC
  • Will then run a fit with MaCh3

22.02.17

For last few weeks I haven't been doing what I thought I'd been doing. I was making fake data with Psyche rather than MaCh3.

Am now making an MaCh3 executable for HPTPC, with the samples/selections needed.

Will then produce the fake data with MaCh3, and fit it.

When I'd fitted this fake data before I was getting the same likelihoods. This is because I was fitting it incorrectly

Prior to that, I'd had an issue with Psyche so copied in a backup version so recreated the fake data to check that hadn't affected anything.

This was identical to how it was before. Below is an example of the data for CC0Pi0P interactions:

HPTPC1_CC0Pi_thetaMom.gifND280_1_CC0Pi_thetaMom.gif

HPTPC1_CC0Pi_Mom1.gifND280_1_CC0Pi_Mom1.gif

HPTPC1_CC0Pi_Theta1.gifND280_1_CC0Pi_Theta1.gif

15.02.17

At collaboration meeting, I attended talks on the ECAL, NIWG, cross-section, BANFF, OA, ND upgrade, and plenary sessions on all aspects of the experiment.

Haven't got to the bottom of why the likelihoods for the fitting of the two fake data sets (HPTPC and ND280) are the same. But there's still a few things I need to check.

--Need to make sure I fully understand the configuration file

--and also what different executables do

Once I'm sure the fittings have worked correctly, will look at how best to vary cross-section model and compare HPTPC/ND280.

--Need to understand all the outputs of MaCh3

01.02.17

Have now got new psyche code working. Added a string as argument of addSelection function, which says which selection is using the sample.

Using this, I recreated Fake Data for HPTPC and ND280 to check it's still the same as before changing the code.

HPTPC_CC0Pi0P_ThetaMom.gif

HPTPC1_CC0Pi_thetaMom.gif

HPTPC_CC0Pi0P_Mom.gifHPTPC1_CC0Pi_Mom1.gif

HPTPC_CC0Pi0P_Theta.gifHPTPC1_CC0Pi_Theta1.gif

New data matches that produced before changing the code, as expected. The same is true for the ND280 fake data.

Then fitted this data with MaCh3, and will compare fits for ND280 and HPTPC.

HPTPC_LogL.gif

ND280_LogL.gif

Looks as expected, with initial burn in phase before levelling out.

Next step is to vary underlying cross-section model and fit the fake data, ultimately to see if HPTPC is able to tell us more about where the models go wrong than ND280.

Need to look at how best to rebin the data so not all in same few bins.

Will talk to people at collaboration meeting next week about how to go about varying the cross-section model.

10.01.17

Still haven't fitted ND280 and HPTPC fake data with MaCh3.

Having trouble with new Psyche code:

-MaCh3 doesn't use internal psyche parameters file, but relies on 1-1 correspondence between selection and a sample.

-With new psyche, multiple selections use the same sample.

Currently mapping selections to samples, to be able to add sample parameter to addSelection function.

Will then run ND280 and HPTPC fake data through MaCh3.

14.12.16

Mark Scott sent over updated psyche code with truth based selection class, along with an example truth spline file.

Went to Imperial on Friday to see Patrick. He helped me get the new psyche code running, and I got the full spline files.

Created fake data for both the ND280 and HPTPC:

ND280_1_CC0Pi_thetaMom.gif

ND280_1_CC0Pi_Mom1.gif

ND280_1_CC0Pi_Theta1.gif

Distributions are consistent with each other, higher density for high cos \theta and low momentum.

HPTPC1_CC0Pi_thetaMom.gif

HPTPC1_CC0Pi_Mom1.gif

ND280_1_CC0Pi_Theta1.gif

Now going to going to fit the data to varying cross-section models.

Hope to get this done by Christmas.

Then push this to far detector.

Now attending MaCh3 meetings, Oscillation Analysis meetings, and ND BANFF meetings.

30.11.16

Playing with MaCh3, looking at the root TTree created from T2K data.

Cross section and flux covariance behave as expected for MCMC:

LogL_nddet_cov.gif

LogL_total_flux_cov.gif

Burn in phase of same length ~10000 iterations.

Similar to energy bins last week, can look at correlation for cross section bins:

xsec_0_1.gif

Appear negatively correlated (?). As bin numbers get further apart, bins become less correlated:

xsec_0_10.gif

xsec_0_25.gif

Still need to investigate other branches in tree.

Currently reading T2K technical notes:
- BANFF fits

- MaCh3

Had meeting with Mark Scott:

- Going to go to Imperial to talk through code with Patrick
- Run Near Detector fit

Get MaCh3 to run truth selection for variations with underlying cross-section models
- If goes well in ND fits, push to oscillation fits

Aim to have test fit done by Christmas

23.11.16

Solved issue running MaCh3. Installing/setting up environment for splines was main issue.

Have now run Near Detector data through MaCh3. The following plot shows the convergence of the Log Likelihood during the Markov process:

The first ~10,000 entries are the burn-in phase. These are discarded, to reduce the effect of the initial choice parameter values. But appears to be behaving as expected.

The following plots are a sanity check, making sure I understand what's going on, and getting used to the syntax of MaCh3 outputs. The first plot shows the energy of the highest filled bin against the energy of the second highest for each simulation. These are highly correlated, as expected:

The next two plots show the energy of the highest filled bin against the 10th, and 100th highest.

Bin_0_9.gif

The correlation has decreased, and decreases further for the 99th bin:

Bin_0_99.gif

If the energy of the highest filled bin is increased, the energy of the second highest bin is also likely to be increased, but the energy of lesser filled bins is not likely to be related.

Currently reading various T2K technical notes and getting to grips with MaCh3.

16.11.16

Went to Imperial a couple of weeks ago to see Patrick and Clarence and install MaCh3:

- Needed to get Root in my directory on scratch, as well as CMT

- Had trouble accessing GSL libraries, as they're in a different place on Royal Holloway servers to Queen Mary, and changing file paths didn't work

- Eventually just pointed to where Asher had reinstalled GSL

- Also got various log ins for T2K intranet, T2KUK wiki account, Slack, EZuce, GitHub repositories etc


Now have got iRods installed to obtain Global Analysis file:

- Have obtained a sample ND280 fit to get used to the systems with

- Currently have an issue running this data with MaCh3


Will then look at getting BANFF:

- Used for MC fit fake data at Near Detector, reproducing Far Detector fake data.

-- WilliamCharlesParker - 16 Nov 2016

Topic attachments
I Attachment Action Size Date Who Comment
GIFgif 0pi1p.gif manage 12.2 K 21 Jun 2017 - 12:01 WilliamCharlesParker  
GIFgif 0piNp.gif manage 12.4 K 21 Jun 2017 - 11:29 WilliamCharlesParker  
GIFgif 130Flux.gif manage 12.3 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 130LogL.gif manage 11.2 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 130Xsec.gif manage 8.3 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 130Xsec0.gif manage 6.6 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 130Xsec10.gif manage 7.0 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 130Xsec15.gif manage 6.3 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 130b50.gif manage 6.7 K 12 Apr 2017 - 13:06 WilliamCharlesParker  
GIFgif 130b_0.gif manage 6.7 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 130b_10.gif manage 7.2 K 12 Apr 2017 - 13:07 WilliamCharlesParker  
GIFgif 1pi1p.gif manage 12.1 K 21 Jun 2017 - 11:29 WilliamCharlesParker  
GIFgif 1piNp.gif manage 11.8 K 21 Jun 2017 - 11:30 WilliamCharlesParker  
GIFgif Bin_0_1.gif manage 13.9 K 23 Nov 2016 - 11:30 WilliamCharlesParker  
GIFgif Bin_0_9.gif manage 16.2 K 23 Nov 2016 - 11:30 WilliamCharlesParker  
GIFgif Bin_0_99.gif manage 17.4 K 23 Nov 2016 - 11:31 WilliamCharlesParker  
GIFgif CC0pi0p_NoBkwdHMNT.gif manage 12.0 K 16 May 2017 - 18:35 WilliamCharlesParker  
GIFgif Fit8LogL.gif manage 4.4 K 29 Mar 2017 - 10:04 WilliamCharlesParker  
GIFgif Fit8LogL_sample.gif manage 5.0 K 29 Mar 2017 - 10:04 WilliamCharlesParker  
GIFgif Flux.gif manage 13.9 K 04 May 2017 - 10:50 WilliamCharlesParker  
GIFgif FluxComparison.gif manage 66.0 K 21 Jun 2017 - 10:44 WilliamCharlesParker  
GIFgif FluxNoHMNT.gif manage 13.9 K 16 May 2017 - 21:46 WilliamCharlesParker  
GIFgif GenieHPTPC.gif manage 16.2 K 10 May 2017 - 11:20 WilliamCharlesParker  
GIFgif Graph1.gif manage 6.1 K 14 Jun 2017 - 13:15 WilliamCharlesParker  
GIFgif HPTPC1_CC0Pi_Mom.gif manage 9.5 K 14 Dec 2016 - 12:32 WilliamCharlesParker  
GIFgif HPTPC1_CC0Pi_Mom1.gif manage 9.2 K 14 Dec 2016 - 13:05 WilliamCharlesParker  
GIFgif HPTPC1_CC0Pi_Theta.gif manage 8.3 K 14 Dec 2016 - 12:32 WilliamCharlesParker  
GIFgif HPTPC1_CC0Pi_Theta1.gif manage 8.1 K 14 Dec 2016 - 13:05 WilliamCharlesParker  
GIFgif HPTPC1_CC0Pi_thetaMom.gif manage 14.1 K 14 Dec 2016 - 12:32 WilliamCharlesParker  
GIFgif HPTPC_CC0Pi0P_Mom.gif manage 8.2 K 30 Jan 2017 - 16:58 WilliamCharlesParker  
GIFgif HPTPC_CC0Pi0P_Theta.gif manage 7.8 K 30 Jan 2017 - 16:59 WilliamCharlesParker  
GIFgif HPTPC_CC0Pi0P_ThetaMom.gif manage 13.1 K 30 Jan 2017 - 16:58 WilliamCharlesParker  
GIFgif HPTPC_LogL.gif manage 10.3 K 01 Feb 2017 - 11:58 WilliamCharlesParker  
GIFgif HPTPC_ND280.gif manage 14.8 K 21 Jun 2017 - 13:26 WilliamCharlesParker  
GIFgif HPTPC_numuCC_0pi0p_5000.gif manage 13.4 K 07 Mar 2017 - 15:20 WilliamCharlesParker  
GIFgif HighP_HighT.gif manage 11.5 K 07 Jun 2017 - 13:17 WilliamCharlesParker  
GIFgif HighP_LowT.gif manage 11.4 K 07 Jun 2017 - 13:17 WilliamCharlesParker  
GIFgif HighP_OrigT.gif manage 11.5 K 07 Jun 2017 - 13:17 WilliamCharlesParker  
GIFgif LogL.gif manage 3.8 K 05 Apr 2017 - 13:15 WilliamCharlesParker  
GIFgif LogL_nddet_cov.gif manage 8.3 K 30 Nov 2016 - 00:26 WilliamCharlesParker  
GIFgif LogL_sample.gif manage 12.5 K 22 Mar 2017 - 10:39 WilliamCharlesParker  
GIFgif LogL_total_flux_cov.gif manage 11.6 K 30 Nov 2016 - 00:26 WilliamCharlesParker  
GIFgif LogLikelihood.gif manage 8.6 K 23 Nov 2016 - 10:46 WilliamCharlesParker  
GIFgif LogLsample.gif manage 4.7 K 05 Apr 2017 - 13:17 WilliamCharlesParker  
GIFgif LogLsmall.gif manage 4.2 K 05 Apr 2017 - 13:16 WilliamCharlesParker  
GIFgif LowP_HighT.gif manage 11.5 K 07 Jun 2017 - 13:16 WilliamCharlesParker  
GIFgif LowP_LowT.gif manage 11.5 K 07 Jun 2017 - 13:17 WilliamCharlesParker  
GIFgif LowP_OrigT.gif manage 11.5 K 07 Jun 2017 - 13:17 WilliamCharlesParker  
GIFgif MEC_O.gif manage 6.4 K 30 May 2017 - 22:46 WilliamCharlesParker  
GIFgif MovedCut.gif manage 12.8 K 10 May 2017 - 11:40 WilliamCharlesParker  
GIFgif ND280_1_CC0Pi_Mom.gif manage 9.5 K 14 Dec 2016 - 12:33 WilliamCharlesParker  
GIFgif ND280_1_CC0Pi_Mom1.gif manage 9.4 K 14 Dec 2016 - 13:05 WilliamCharlesParker  
GIFgif ND280_1_CC0Pi_Theta.gif manage 8.2 K 14 Dec 2016 - 12:33 WilliamCharlesParker  
GIFgif ND280_1_CC0Pi_Theta1.gif manage 8.0 K 14 Dec 2016 - 13:05 WilliamCharlesParker  
GIFgif ND280_1_CC0Pi_thetaMom.gif manage 14.1 K 14 Dec 2016 - 12:33 WilliamCharlesParker  
GIFgif ND280_LogL.gif manage 10.2 K 01 Feb 2017 - 13:16 WilliamCharlesParker  
GIFgif NoCut.gif manage 12.2 K 10 May 2017 - 11:58 WilliamCharlesParker  
GIFgif OrigP_HighT.gif manage 11.5 K 07 Jun 2017 - 13:16 WilliamCharlesParker  
GIFgif OrigP_LowT.gif manage 11.5 K 07 Jun 2017 - 13:17 WilliamCharlesParker  
GIFgif RatioPlot.gif manage 9.4 K 14 Jun 2017 - 12:55 WilliamCharlesParker  
PNGpng Screen_Shot_2017-06-14_at_13.36.26.png manage 651.0 K 14 Jun 2017 - 13:39 WilliamCharlesParker  
PNGpng Screen_Shot_2017-06-14_at_13.42.06.png manage 44.9 K 14 Jun 2017 - 13:42 WilliamCharlesParker  
JPEGjpeg Table.jpeg manage 65.8 K 12 Apr 2017 - 13:08 WilliamCharlesParker  
PDFpdf TransverseVariableSpectra.pdf manage 795.3 K 28 Jun 2017 - 10:03 WilliamCharlesParker  
PNGpng TransverseVariables.png manage 99.2 K 14 Jun 2017 - 12:52 WilliamCharlesParker  
GIFgif Xsec.gif manage 12.3 K 04 May 2017 - 10:50 WilliamCharlesParker  
GIFgif XsecComparison.gif manage 54.7 K 21 Jun 2017 - 10:44 WilliamCharlesParker  
GIFgif b10.gif manage 5.5 K 05 Apr 2017 - 13:15 WilliamCharlesParker  
GIFgif b_20.gif manage 6.4 K 05 Apr 2017 - 13:15 WilliamCharlesParker  
GIFgif cc0pi0p.gif manage 12.0 K 07 Jun 2017 - 13:02 WilliamCharlesParker  
GIFgif cc0pi1p.gif manage 10.4 K 03 May 2017 - 17:18 WilliamCharlesParker  
PDFpdf dP.pdf manage 89.0 K 28 Jun 2017 - 10:35 WilliamCharlesParker  
PDFpdf dalpha.pdf manage 78.2 K 28 Jun 2017 - 10:24 WilliamCharlesParker  
PDFpdf dphi.pdf manage 88.4 K 28 Jun 2017 - 10:24 WilliamCharlesParker  
GIFgif fit8b_0.gif manage 6.9 K 29 Mar 2017 - 10:05 WilliamCharlesParker  
GIFgif fit8b_10.gif manage 7.1 K 29 Mar 2017 - 10:12 WilliamCharlesParker  
GIFgif fit8b_20.gif manage 7.2 K 29 Mar 2017 - 10:05 WilliamCharlesParker  
GIFgif fit8xsec_0.gif manage 7.6 K 29 Mar 2017 - 10:05 WilliamCharlesParker  
GIFgif fit8xsec_10.gif manage 9.2 K 29 Mar 2017 - 10:04 WilliamCharlesParker  
GIFgif fit8xsec_20.gif manage 6.8 K 29 Mar 2017 - 10:04 WilliamCharlesParker  
GIFgif flux_scale01.gif manage 14.0 K 17 May 2017 - 09:46 WilliamCharlesParker  
PNGpng my_first_hsub_r01164033_e099_c0.png manage 117.7 K 14 Jun 2017 - 13:21 WilliamCharlesParker  
GIFgif ndd_350.gif manage 7.0 K 07 Mar 2017 - 15:46 WilliamCharlesParker  
GIFgif noise_exposure_1x1.gif manage 5.1 K 28 Jun 2017 - 13:12 WilliamCharlesParker  
GIFgif noise_exposure_2x2.gif manage 5.6 K 28 Jun 2017 - 14:21 WilliamCharlesParker  
GIFgif numuCC0pi0p_1.gif manage 12.6 K 03 May 2017 - 17:08 WilliamCharlesParker  
GIFgif numuCC0pi0p_projY.gif manage 6.3 K 03 May 2017 - 17:22 WilliamCharlesParker  
GIFgif pLow_Ratio.gif manage 10.5 K 14 Jun 2017 - 12:45 WilliamCharlesParker  
PNGpng r1150013.png manage 384.6 K 30 May 2017 - 23:40 WilliamCharlesParker  
GIFgif scale_noScaleOverlay_MAQE.gif manage 6.0 K 17 May 2017 - 09:55 WilliamCharlesParker  
GIFgif scale_noScaleOverlay_MARES.gif manage 5.9 K 17 May 2017 - 09:55 WilliamCharlesParker  
GIFgif scale_noScaleOverlay_NCCoherentNorm.gif manage 7.7 K 17 May 2017 - 09:55 WilliamCharlesParker  
GIFgif scale_noScaleOverlay_b0.gif manage 6.5 K 17 May 2017 - 09:56 WilliamCharlesParker  
GIFgif scale_noScaleOverlay_b1.gif manage 6.6 K 17 May 2017 - 09:55 WilliamCharlesParker  
GIFgif scale_noScaleOverlay_b2.gif manage 5.9 K 17 May 2017 - 09:55 WilliamCharlesParker  
GIFgif scale_noScaleOverlay_b3.gif manage 5.8 K 17 May 2017 - 09:56 WilliamCharlesParker  
GIFgif xsec0.gif manage 6.3 K 05 Apr 2017 - 13:16 WilliamCharlesParker  
GIFgif xsec20.gif manage 7.1 K 05 Apr 2017 - 13:16 WilliamCharlesParker  
GIFgif xsecNoHMNT.gif manage 12.8 K 16 May 2017 - 21:46 WilliamCharlesParker  
GIFgif xsec_0_1.gif manage 17.3 K 30 Nov 2016 - 00:26 WilliamCharlesParker  
GIFgif xsec_0_10.gif manage 16.2 K 30 Nov 2016 - 00:26 WilliamCharlesParker  
GIFgif xsec_0_25.gif manage 16.5 K 30 Nov 2016 - 00:26 WilliamCharlesParker  
GIFgif xsec_scale01.gif manage 13.5 K 17 May 2017 - 09:46 WilliamCharlesParker  

Physics WebpagesRHUL WebpagesCampus Connect • Royal Holloway, University of London, Egham, Surrey TW20 0EX; Tel/Fax +44 (0)1784 434455/437520

Topic revision: r39 - 28 Jun 2017 - WilliamCharlesParker

 
This site is powered by the TWiki collaboration platformCopyright © 2008-2017 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding RHUL Physics Department TWiki? Send feedback