8/20/99 - RLC First 5 Diracs were unpacked, given serial numbers "PROD 001" thru 005. None can be reliably programmed. Scoping the bitblaster clock (TCK) we can see an obvious bounce on the rising edge. Currently 003 was the only one to eventually be completely programmed and it passed all static tests and FS0 (1 loop at least). The programs on this board are in an unknown state now. We are using the byteblaster (it is much faster). Currently our control, PP1, also does not program reliably.

Since front panels no longer belong to a board, here is the conversion to what is written on the board next to the tracking input:
PP1 = "Serial #0007 PP1"
PP2 = "Serial #0006 PP2"
PP3 = "Serial #0008 PP3"
PP4 = "PP4"
prototype with "3 BAD" and "11 Bad" written in red ink on front panel and 6 jumper wires = "proto 2"
prototype with pencil labels including "#13" on front panel = "Serial #0005"
pprototype with "dirac 2" sticker on front panel, blue jumper wire, = "proto 3"
prototype with "dirac 1" sticker on front panel, white jumper wire, = "proto 1"

8/23/99 - Pedro
Mary installed newest version of Altera 9.3 today. Starting use this one as default from now on.
Programmed 1 Prototype and 1 Prepoduction 4 times each, full download, with ByteBlaster. Occasional errors on ERROR chip (1 out of 4 times). Reprogramming the ERROR chip alone solved the problem.

Production 002 was succesfully programmed. Again error on ERROR chip. Got rid of it by programming it alone. Passes all static tests at once. Passes all FullSpeed tests below at once.

Production 003 was succesfully programmed after about 10 trials on the ETInp(bigAdder) and ERROR chips. But fails Static test DAQ registers and static test with VME clk

8/24/99 - Pedro
Started to work around the TSI trace to make it slower on P001. On this board, all the chips can be programmed reliably except the ERROR chip.
On P001 the FCT chip was replaced with a HCT. No improvement.
On P001 several RC's were added along the TCK line: at the source, on the error chip and at U33 (termination of the other side of this line). No improvement.
RC added at TDI of error chip. No improvement.
Ground and power pins of ERROR checked for DC level and AC spikes. Nothing found.
TCK signals of Pre-Production look about the same as for Production.
TMS signals looked OK in scope. Meaning stable on rising edge of clock, with no spikes during programming...

On several boards (3 out of 13) the chain is NOT detected at all...
Mircea talked with Altera and had the idea of using a 50 pf capacitor on the TMS line. On one board that allowed the ERROR chip to be programmed, but NOT on another.

So summary:
One other board had all chips succesfully programmed, and passes all static and full-speed tests.
That makes 2 boards approved, out of 13
6 Boards have 14 chips successfully programmed at once. The one missing is the error chip.
2 boards have the chain broken.
1 board has error on Towmask (a first, never saw that before on any production or prototype).
1 board has CSUM chip instead of ETSUM...
1 board I succesfully downloaded the WRONG error program and blew the chip...
Idea for tomorrow is to keep ON working on noise in the TMS line...

8/25/99 - Pedro
Mircea and Mary found out in the Altera page, that versions 9.24 have a bug when programming our chips(7128S). They say it's fixed in 9.3, and give a version (9.26) that supposudely fixes the chips that were bad. Note that not only 9.24 may not program them correctly, also can actually damage them in a way that is only recoverable using 9.26. At least that's our interpretation of the Altera info.

So here's what we found out so far:
-version 9.3 is WRONG, regardless of Altera claiming its right. That was the version we programmed the boards with yesterday.
-When PRE-production was made, the change in the HEART chip was compiled already with 9.24. So it seems like during PRE-production, initially the chips were successfully downloaded with the good 9.21, and then the Heart chip loaded with the bad 9.24. It seems to me that if you were once able to program one chip using a good version, then even if you do it again with a bad version, it's ok. This would explain why the pre-production boards were almost always OK, even when reprogrammig them with 9.3 So this problem was here even at pre-production, but we were so unlucky we missed it then... (moreover, version 9.21 had already been removed from the disk...)
-It's not clear if 9.26 will actually fix the chips (see below, though). It's also not clear how the 9 (nine !) boards that were erroneously programmed were affected.
-On the 2nd box of Diracs, ALL 11 boards, were successfully programmed with version 9.21, using e:\users\cdf\toolatedirac\copiedtotestslew\control\full_download.jcf and byteblaster.
- On this 2nd box, No board gave the broken chain error. Last day, some of the boards gave broken chain immediatly, even without being programmed. Mircea had the idea that even when one is just detecting the chain, you are already doing something on the chips. So perhaps those chains were also broken by simple chain detecting with the bad 9.3 version.
- So the programming problem is solved, thanks to Mircea and Mary...(a.k.a. M & M :-) ). no RCs, no cludges, no HCTs!

2 boards (P004 and P006) were resurected successfully, see below. The others 6 don't give a chance, since I could never detect the chain, even trying many times (about 10) on each board. (power on/off to cool boards, between trials)
Board P001 was reprogrammed once with 9.26 version, the all chain, and then it passed all tests. I just noted that the 3 different tests used don't test that many different combinations of masking...Can also use test FSA6, but this one has the same masking as FSA0. Can start to use FSA4 which masks off almost all towers, but also reprograms the FRAMs on the ADmems. This is feasible, if we get the PowerPC going...
Did the same on Board P005. But had to reprogram the ERROR chip individually many times with 9.26. Noticed that if (After) programming, I verify ERROR, it's never good. Also many times the red ERROR led did not come on, (as it should on power on) even after reprogramming.(Also tried to reprogram the all thing with 9.21)
It fails AL_FIFO, CSUM, and DAQ (and obviously VME_CLK).
On board 17, also tried to reprogram, and here things went bad: now the chain is broken...Here I believe it's now broken in VME_interface
So summary:
All 11 boards in 2nd box were programmed OK.
Only 2 out of these 11 don't pass tests.
We have a total of 13 good boards. Plus PROD001, which is now also OK, but that we might want to take out the RCs cludged to it.
On 6 boards the chain is defenetly broken. Must try to replace the chips, starting with the ERROR chip...
4 boards programmed OK, but with errors on test. Out of these tried more on PROD005, and things look bad on this one...
Score: 14-10...

P.S. There were 2 more boards with CSUM chips instead of ETSUM.

8/26/99 - Pedro
Opened 3rd box, with 6 boards. 1 board has CSUM chip instead of ETSUM. Did NOT touched that one. All remaining 5 programmed and tested OK!
So it defenetly seems to me that Promex is sending us good boards, except the ones with CSUM replaced by ETSUM. On those (4 up to now) we already tried to replace ETSUM chip, and all of them have errors around the CSUM (yes, CSUM) section. My current guess is that on those boards, not only ETSUM chips are bad, also something was bad on CSUM, despite the chips looking good visually. Keep working on this...
All 4 CSUM chips replaced on P005. Still same error.
Error seems to be on ALFifo. For P005, for tower9 (that's around U177, the towtot and lsb), and for board P019, for tower6 (U176, for EM, the LSB 2 bits)
Looked at signal in scope for P019, output of U176 looks good,(meaning is NOT stuck low, as I would expect) although with some spikes on the line. Connections of U176 to U28 and to U75, U81 measured all OK with Ohmeter. Keep working on this tomorrow
Board P003 was reprogramed (see below) and is now OK.
Score: 20-9. (we're winning):
20 tested OK. 6 damaged with MaxPlus 9.3.
1 (P017) (also) damaged with 9.26 when trying to get rid of error...
2 Programmed OK, but failing tests.

8/27/99 -Pedro
Worked on board P019, and solved the problem by identifying 2 pins in U28, that were making contact. (See below). Board now working.
Score: 21-8.

8/30/99 -Pedro
Reprogrammed with 9.21boards that were touched with 9.3 or 9.26. To my surprise all worked fine. Well, all except P003, which at least once after being reprogrammed failed some tests. Either way, that one board finally passed all tests, and is still in the OK section.
Score as before: 21-8.

8/31/99 -Pedro
When using PP1 and PP3 (preproduction) to test the new Aux cards, found error on Front-Panel output, channel 3. I had tested these boards OK, before !...

9/1/99 -Pedro
Mircea received the Altera engineers and proved them that 9.21 is OK (by successfully programming a board) and 9.3 is wrong (by being unable to program that same board)
4 boards had the ERROR chips replaced. On 3 of them, the all chain was programmed successfully and tested OK.
The 4th one the ERROR chip was replaced but still the all programming couldn't be done.
Keep working on this...
Score: 24-6
24 good.
1 (P005) programmed OK, but error on static tests.
4 still not programmed OK.
1 with ETSUM chip wrong, not yet replaced.

9/2/99 -Pedro
Replaced bad ETSUM chip on board P030. Board programmed OK (twice) with 9.21 and all tests OK.
Score: 25-5
25 good.
1 (P005) programmed OK, but error on static tests.
4 still not programmed OK.

9/20/99 - Pedro
Received 20 more Diracs.
Found the all system NOT working after all the things that should have NEVER been done on THIS crate at this time: Testing aux cards and P3 backplane.
Found board p030, a previously tested good one in the same crate as the 5 bad ones.
Boards P030, P001, P002, P028 all of them previously tested OK, all failed the Full Speed tests and the VME clk Static test.
After 2 hours of work on the TSCRs and fixing a discrepancy in the FullSpeedAhead code (that was NOT there in 9/2/99), all previous 4 boards pass the FullSpeed tests, but still fail the static VME clk test (and only this one). Most probably (new) software bugs...
Not really: The new P3 pushes the boards out, such that P1, P2 are not all the way in. Namely P1, the upper one, where SYSCLK is. Hammering on the Dirac boards, like I used to do 1 year ago, "solves" the problem.
3 Boards programmed and tested OK.
Score: 28-5

9/21/99 Pedro
3 boards tested, one bad, 2 good.
SYSCLK problem is defenetly related to "hammering". Either P1 connection, or MVME board problem, or both.
Score: 30-6

9/22/99 Pedro + Peter
P3 backplane and CPU changed (to a Power PC) and still problem on SYSCLK. Most probably a bad adapter board.
9 more boards tested, 1 with errors in static tests due to tower errors. (5 more to go out of this batch). All 20 boards were programmed OK.
A new batch of 20 arrived today.
Score: 38-7

9/23/99 Pedro
Brought new adapter for cpu from Fermilab. SYSCLK now stable.
One more board tested ok. 4 to go out of this batch.
Score: 39-7

9/24/99 Peter + Pedro
20 boards programmed OK.
12 more boards tested OK.
Score: 51-7

9/25/99 Pedro + Peter
10 boards test OK, 1 board fails.
Score: 62-8

9/29/99 Pedro
CONTROL program changed (in sport: e:\users\cdf\toolatedirac\copied to test slew\newcontrol3\control ).
ID_PROM register now gives: 00XX 005 DIRAC
I think there is no way of getting the serial number read from the switches into this ID_PROM word, as those lines are NOT connected to the CONTROL chip. Moreover the CONTROL chip is almost full.
One board (P0034) that was programmed OK before, now can't be programmed.
Mircea and I found that board P013 can't be programmed on the crate (chain is not even recognized), but it works fine using the small power supplies.
So when it comes to programming, there seems to be an issue of noise of the power supplies feeding the crate. We checked that it does not depend on the slot, and programming on the crate does work for most boards.
Board P013 now passes all tests.
Score: 63-7

10/4/99 Peter + Pedro
Just a check on the current score:
Peter: Score: 85-10 (as of testing of dirac 92)
Board P007 with Brain chip replaced, passes all tests.
Final score: 91-9.
91 tested OK. 16 out of these, with stiffeners and switches glued, ready to go.
5 not programmable.
4 programmed but fail tests.
8 inside box, not yet serialized.

10/5/99 Pedro + Peter
Note: All boards have a routing problem that causes the following problem with the board serial number switch (S1):
setting bit 6 on the switch sets bit 7 on the output
setting bit 7 on the switch sets bit 8 on the output
setting bit 8 on the switch sets bit 6 on the output
(starting bit count from 0, so switch 7 sets bit 6, etc.)
This affects boards number >63. They were set to give the correct output.

4 boxes of 16 diracs each have been packed.
Received the last 3 Diracs
3 more diracs programmed and all tested OK
Score: 94-9
Counting with the 4 pre-production, we are already over 96 !!!

10/6/99 Pedro
Measured the currrent drawn by Dirac to never exceed 3.5-3.6 Amps.
Harold points out that the current spikes higher when the crate is powered on, and that that depends on the power supply also. So keep on using 10 Amp fuses for Dirac seems to be ok.
Score:99-12 (not counting the pre-production)
Score:103-12 (with 4 pre-production)
96 boards in 6 boxes all tested, with switches glued, stiffeners and front-panels, ready to go to fnal !
Family photo taken, with some Porto and Coke along. Sorry if you weren't there!...
The 12 bad ones include 5 that can't be programmed, 6 that have bit errors, and 1 that has 1 of the small plastic guides coming out of P3 broken. Tried 9.26 on some of the 5 not programmable, with no success.
Will work on the 6 with bit errors in the next days.

10/7/99 Pedro

96 boards sent to Fermilab, showed at the DAQ meeting and now in Peter's office !

10/8/99 Pedro
Mircea found (and fix) a problem with P3 connectors on 2 boards (P017 and P108).
Those 2 boards are now OK.
Reprogrammed board P037. Board OK now.
Score: 106-9
Out of the 9 bad ones, 7 can't be programmed, 1 has a P3 connector broken, and other fails tests 20% of the time, due to a clock distribution problem (P105).
We have 10 spares, all with stiffeners and switches glued, ready to go to FermiLab as reinforcements.
In case we need them, I want to keep the pre-productions (P112-P115) here, if possible.

10/8/99 Pedro
OH GLORIOUS DAY !
Following the work on the last few weeks, the last bad board (P078) is now working and all of the 115 boards have now been programmed, passed the checkout tests and had switches glued and stiffeners put.
There are 3 boards (P080, P088, P101) that have the plastic thingie on the P3 connector broken. And P011 is working fine, but should only be used as a last resort, since the brain chip CANNOT be replaced, as the pads are coming out.
See you (this logbook) never again ?... :-)

Basic tests

Delivered: 13
Passed basic tests: 0
Moved to FNAL permanently: 0
No longer tracked here: 0

Dirac 1

Dirac 2

Dirac 3

Dirac 4

Dirac 5

Dirac 6

Dirac 7

Dirac 8

Dirac 9

Dirac 10

Dirac 11

Dirac 12

Dirac 13

Dirac 14

Dirac 15

Dirac 16

Dirac 17

Dirac 18

Dirac 19

Dirac 20

Dirac 21

Dirac 22

Dirac 23

Dirac 24

Dirac 25

Dirac 26

Dirac 27

Dirac 28

Dirac 29

Dirac 30

Dirac 31

Dirac 32

Dirac 33

Dirac 34

Dirac 35

Dirac 36

Dirac 37

Dirac 38

Dirac 39

Dirac 40

Dirac 41

Dirac 42

Dirac 43

Dirac 44

Dirac 45

Dirac 46

Dirac 47

Dirac 48

Dirac 49

Dirac 50

Dirac 51

Dirac 52

Dirac 53

Dirac 54

Dirac 55

Dirac 56

Dirac 57

Dirac 58

Dirac 59

Dirac 60

Dirac 61

Dirac 62

Dirac 63

Dirac 64

Dirac 65

Dirac 66

Dirac 67

Dirac 68

Dirac 69

Dirac 70

Dirac 71

Dirac 72

Dirac 73

Dirac 74

Dirac 75

Dirac 76

Dirac 77

Dirac 78

Dirac 79

Dirac 80

Dirac 81

Dirac 82

Dirac 83

Dirac 84

Dirac 85

Dirac 86

Dirac 87

Dirac 88

Dirac 89

Dirac 90

Dirac 91

Dirac 92

Dirac 93

Dirac 94

Dirac 95

Dirac 96

Dirac 97

Dirac 98

Dirac 99

Dirac 100

Dirac 101

Dirac 102

Dirac 103

Dirac 104

Dirac 105

Dirac 106

Dirac 107

Dirac 108

Dirac 109

Dirac 110

Dirac 111

Dirac 112

Dirac 113

Dirac 114

Dirac 115