Page 1 of 1

Lambda Delay TunerStudio Log Parser

Posted: Sun Jan 01, 2017 5:24 pm
by luftvagon
Got bored, decided to write a "Lambda Delay" estimator using parsing of your TunerStudio logs. The more records, the longer the log, the better.

If anybody has a very long TunerStudio log, and you think your Lambda Delay is correct, please let me take a look at your log file to see if numbers will match close to the script output.

Usage:

Code: Select all


$ l_delay.gawk Downloads/2016-12-27_18.01.04.msl 
F:1656:S:1657:T:1662 Delay: 387ms RPM: 1180 MAP: 64
F:1657:S:1658:T:1662 Delay: 322ms RPM: 1196 MAP: 64
F:1666:S:1667:T:1670 Delay: 200ms RPM: 2128 MAP: 65
F:1680:S:1681:T:1686 Delay: 334ms RPM: 1502 MAP: 65
F:1719:S:1720:T:1724 Delay: 270ms RPM: 2093 MAP: 65
...
Requirements:
  • GNU Awk
Limitations:
  • Single Wide Band Sensor
  • Single Channel Batch Fuel Delivery
Change Log:
  • 00001a

Code: Select all


Split "events" ahead into PW, and AFR. 
Spit out event line # for first,second,and third data point for log reference lookups.
  • 00001

Code: Select all


Initial Release

Re: Lambda Delay TunerStudio Log Parser

Posted: Sun Jan 01, 2017 5:25 pm
by luftvagon
Code:

Code: Select all


#!/usr/bin/awk -f 
# awk script to figure out lambda delay based on changes to pw, followed by afr changes logged within tunerstudio mlv logs.
# version 00001a

BEGIN {
# Field Separator; either \t or , for CSV
FS="\t"

# How many PW events ahead to look for deltas
pw_events_ahead=1

# How many AFR events ahead to look for deltas
afr_events_ahead=5

# Reject events with CLT below
clt_reject_below=170

# Delta PW (10%)
delta_pw = 90

# Delta AFR (15%)
delta_afr = 85
}

function cmp_num_idx(i1, v1, i2, v2)
{
# numerical index comparison, ascending order
return (i1 - i2)
}

/Time/ {
q=split($0,a,"\t")
for (v=1;v<=q;v++) {
c[a[v]]=v
}
}

/^[0-9]/ {
q=split($0,a,"\t")
track[NR]["TIME"] = a[c["Time"]]
track[NR]["RPM"] = a[c["RPM"]]
track[NR]["MAP"] = a[c["MAP"]]
track[NR]["AFR"] = a[c["AFR"]]
track[NR]["PW"] = a[c["PW"]]
track[NR]["CLT"] = a[c["CLT"]]
}

END {
PROCINFO["sorted_in"] = "cmp_num_idx"
for (first_data_point in track) {
if (track[first_data_point]["CLT"] >= clt_reject_below) {
for (second_data_point = first_data_point; second_data_point <= first_data_point+pw_events_ahead; second_data_point++) {
if ((track[second_data_point]["PW"] > delta_pw/100) && (track[first_data_point]["PW"] > 100/delta_pw) ) {
deltapw = track[first_data_point]["PW"]/track[second_data_point]["PW"]
} else {
deltapw = 1
}
if ((deltapw < delta_pw/100) || (deltapw > 100/delta_pw)) {
for (third_data_point = second_data_point; third_data_point <= second_data_point+afr_events_ahead; third_data_point++) {
if ((track[second_data_point]["AFR"] > 0.1) && (track[third_data_point]["AFR"] > 0.1) ) {
deltaafr = track[third_data_point]["AFR"]/track[second_data_point]["AFR"]
} else {
deltaafr = 1
}
if ((deltaafr < delta_afr/100) || (deltaafr > 100/delta_afr)) {
if ((track[third_data_point]["TIME"]-track[second_data_point]["TIME"] < 1 ) && (track[second_data_point]["RPM"] > 0)) {
printf "F:%d:S:%d:T:%d Delay: %dms\tRPM: %d\tMAP: %d\n", first_data_point, second_data_point, third_data_point, (track[third_data_point]["TIME"]-track[second_data_point]["TIME"])*1000 , track[second_data_point]["RPM"] , track[second_data_point]["MAP"]
}
break
}
}
break
}
}
}
}
}
Alternative:

Find 0 PW, and seek for "freeair" AFR; display delta between the two events.

Code: Select all


#!/usr/bin/awk -f 
# awk script to figure out lambda delay based on changes to pw, followed by afr changes logged within tunerstudio mlv logs.

BEGIN {
# Field Separator; either \t or , for CSV
FS="\t"

# Maximum events to seek ahead
events_ahead = 15

# Controller Freeair AFR
freeair_afr = 22.3

# Controller Power-on AFR
poweron_afr = 7.5

# Reject events with CLT below
clt_reject_below=120
}

function cmp_num_idx(i1, v1, i2, v2)
{
# numerical index comparison, ascending order
return (i1 - i2)
}

/Time/ {
q=split($0,a,"\t")
for (v=1;v<=q;v++) {
c[a[v]]=v
}
}

/^[0-9]/ {
q=split($0,a,"\t")
track[NR]["TIME"] = a[c["Time"]]
track[NR]["RPM"] = a[c["RPM"]]
track[NR]["MAP"] = a[c["MAP"]]
track[NR]["AFR"] = a[c["AFR"]]
track[NR]["PW"] = a[c["PW"]]
track[NR]["CLT"] = a[c["CLT"]]
}

END {
PROCINFO["sorted_in"] = "cmp_num_idx"
for (first_data_point in track) {
if ((track[first_data_point]["CLT"] >= clt_reject_below) && (track[first_data_point]["AFR"] > poweron_afr)) {
if (track[first_data_point]["PW"] < 1) {
found = 0
for (second_data_point = first_data_point+1; second_data_point <= first_data_point+events_ahead; second_data_point++) {
if ((track[second_data_point]["AFR"] == freeair_afr) && (track[first_data_point]["AFR"] < freeair_afr) && (track[first_data_point]["AFR"] > poweron_afr)) {
if (found == 0) {
printf "Event: %d-%d \tDelta: %dms \tPW: %3.2f \tMAP: %d \tRPM: %d \tAFR: %3.1f \t| \tPW: %3.2f \tMAP: %d \tRPM: %d \tAFR: %3.1f \n" , first_data_point, second_data_point, 1000*(track[second_data_point]["TIME"]-track[first_data_point]["TIME"]) , track[first_data_point]["PW"], track[first_data_point]["MAP"], track[first_data_point]["RPM"], track[first_data_point]["AFR"], track[second_data_point]["PW"], track[second_data_point]["MAP"], track[second_data_point]["RPM"], track[second_data_point]["AFR"]
found = 1
}
}
}
}
}
}
}

Re: Lambda Delay TunerStudio Log Parser

Posted: Sun Jan 01, 2017 8:36 pm
by MarioVelotta
luftvagon wrote:If anybody has a very long TunerStudio log, and you think your Lambda Delay is correct, please let me take a look at your log file to see if our numbers match. I'll be posting the script in the 2nd thread, after I do some more testing.
Hey Luftvagon, way to go! Can you teach me how to code? :)

How would our numbers match? Each install is going to be different due to countless differences from engine to engine. I've got some long logs with data from the 6 02 sensors I have in my exhaust right now :roll: :mrgreen: The Major 4 are just off the heads about 4", the 5th is just before the collector on a type 1 sidewinder type exhaust and the 6th is about 4" further but after the collector. This was actually to test the internal lambda controller in my new ECU's against the LC2 that was in the 5th position.

Re: Lambda Delay TunerStudio Log Parser

Posted: Sun Jan 01, 2017 9:17 pm
by luftvagon
Hey Mario,

The number(s) to compare would be your lambda delay (probably calculated using a different method/guessing/or default) versus what the your log produces. I don't have any of my own logs to try it on, so I will go for a long drive on Tuesday. Currently, I was testing this on someone else's 5 MB log file, and it got some good data points.

Here is what the program/parser does:
- I take each event in the log that has CLT>130F (adjustable) (event1)
- For each event, I look for subsequent events that have PW delta __ % (configurable) (event2)
- and then, I follow event2 subsequent AFR delta __%. (event3)
- If event3-event2 AFR delta % is met, spit out event3-event2 time delta, followed by event2's RPM, and event2 MAP.

In theory, this sounds "close" to the magic numbers we need. One could go further to average this, because there would be many data points.

Re: Lambda Delay TunerStudio Log Parser

Posted: Mon Jan 02, 2017 1:48 pm
by luftvagon
I wrote another version that will only look for PW = 0, and then look for subsequent AFR DROP -- it may be more helpful:

Code: Select all


$ l_delay_3.gawk Downloads/2016*.msl | more
Event: 1812-1813 Delta: 62ms PW: 0.00 MAP: 17 RPM: 2156 AFR: 16.3 | PW: 0.00 MAP: 17 RPM: 2149 AFR: 22.3
Event: 2153-2154 Delta: 78ms PW: 0.00 MAP: 17 RPM: 2054 AFR: 21.4 | PW: 0.00 MAP: 18 RPM: 2023 AFR: 22.3
Event: 2542-2543 Delta: 93ms PW: 0.00 MAP: 18 RPM: 1937 AFR: 20.0 | PW: 0.00 MAP: 18 RPM: 1950 AFR: 22.3
Event: 2629-2631 Delta: 125ms PW: 0.00 MAP: 22 RPM: 1499 AFR: 22.3 | PW: 0.00 MAP: 25 RPM: 1451 AFR: 8.4
Event: 2630-2631 Delta: 61ms PW: 0.00 MAP: 23 RPM: 1488 AFR: 22.3 | PW: 0.00 MAP: 25 RPM: 1451 AFR: 8.4
Event: 2631-2632 Delta: 78ms PW: 0.00 MAP: 25 RPM: 1451 AFR: 8.4 | PW: 1.58 MAP: 28 RPM: 1470 AFR: 7.5
Event: 3601-3602 Delta: 62ms PW: 0.00 MAP: 17 RPM: 2163 AFR: 21.4 | PW: 0.00 MAP: 18 RPM: 2154 AFR: 22.3
Event: 10211-10212 Delta: 62ms PW: 0.00 MAP: 15 RPM: 2557 AFR: 22.2 | PW: 0.00 MAP: 15 RPM: 2565 AFR: 22.3
Event: 11536-11537 Delta: 62ms PW: 0.00 MAP: 18 RPM: 1887 AFR: 22.0 | PW: 0.00 MAP: 19 RPM: 1880 AFR: 22.3

Re: Lambda Delay TunerStudio Log Parser

Posted: Mon Jan 02, 2017 3:49 pm
by Piledriver
Nice work!

I'll give it a try later this week, waiting on some brake bits.

Re: Lambda Delay TunerStudio Log Parser

Posted: Mon Jan 02, 2017 4:27 pm
by MarioVelotta
When it looks for 0.00 PW, are you then looking for the Lambda Feedback to reach freeair or just change from current?

Re: Lambda Delay TunerStudio Log Parser

Posted: Mon Jan 02, 2017 4:46 pm
by luftvagon
Mario, I can make it either or configurable.. What would be freeair? 22.3? Guess that value would be controller specific.

awk code posted above.

Re: Lambda Delay TunerStudio Log Parser

Posted: Tue Jan 03, 2017 11:01 pm
by MarioVelotta
Anyway I can run this on a PC without linux?

Re: Lambda Delay TunerStudio Log Parser

Posted: Wed Jan 04, 2017 2:12 am
by Piledriver
MarioVelotta wrote:Anyway I can run this on a PC without linux?
More work than you would probably like.
Installing the gnu tools compiled for windows has always been iffy, if available at all.

That's why MS is working on a compatibility layer and has ported bash to WinX...
bash makes powershell seem like command.com...

Boot from a USB stick of DVD, recent Knoppix DVD has all of the tools you need, and you can "install" more packages w/o installing to you HDD. You can even install TS etc. Works fine, even with just a "persistent" Live install to USB.

You can even install fully to a large USB stick or drive, works fine, although Knoppix isn't really intended to be a real distro.
Mint 18/kde or Mageia 5.1 are probably better choices, and can also be installed to USB.
(Mageia 6 will be out RSN)

Re: Lambda Delay TunerStudio Log Parser

Posted: Thu Jan 05, 2017 8:45 am
by luftvagon

Re: Lambda Delay TunerStudio Log Parser

Posted: Wed Nov 08, 2017 2:01 pm
by RHough
Adding a self learn Lambda delay function is something Tuner Studio should be doing. :-) All the information is there just need TS to look at the data stream and create a Lamda Delay map to work with VE auto tune. Parsing logs either manually or with a third party app is lots of work to make the auto tune function work correctly. I think what we need to look at is PWdot vs Lambda Response. Seeing auto tune adjust the same cells multiple times is odd. I assume that this behaviour is dependant on Lambda delay? Looking at PW = 0 and lambda response is a closed throttle delay time. When I look for the PW 0 to + and lambda response from full lean I get different numbers than when I look for PW + to 0 and lambda response. I think (dangerous) that when the delay is correct I should see longer periods of no EGO corrections? Having to set reference points in the fuel map to then parse logs looking for response times and guestimate the delay table is very time consuming. ;-) IIRC my WB samples at 12hz so output is every 83.3ms? I assume length from exhaust valve to O2 sensor is one factor, exhaust system volume to O2 sensor is another, and volume of exhaust at RPM & load is the last. What we need is to know what the exhaust volume is then calculate the number of engine cycles it takes for that volume to reach the sensor. That should give an RPM based first approximation of lambda delay? Then there is a +/- timing error from the lambda sample rate? Seems to me that =/- 83ms is as close as I can get?

Re: Lambda Delay TunerStudio Log Parser

Posted: Wed Nov 08, 2017 3:26 pm
by Piledriver
At least the table in tunerstudio is only 3x3.

MS3 has a much larger delay (8x8?) table built into the firmware that needs populated.

The main thing is get it reasonably close, combustion itself has a lot of chaos, so there will always be noise.

The "default" delay values are for an LSx v8 with short cast iron manifolds with the O2 sensor a few inches from an exhaust port, so any workable boxer engine exhaust with collector mounted sensor will likely have a LOT more delay.

~Anything reasonably close will bring an improvement in VEAL performance.

Re: Lambda Delay TunerStudio Log Parser

Posted: Wed Nov 08, 2017 6:25 pm
by RHough
Piledriver wrote:
Wed Nov 08, 2017 3:26 pm
At least the table in tunerstudio is only 3x3.

MS3 has a much larger delay (8x8?) table built into the firmware that needs populated.

The main thing is get it reasonably close, combustion itself has a lot of chaos, so there will always be noise.

The "default" delay values are for an LSx v8 with short cast iron manifolds with the O2 sensor a few inches from an exhaust port, so any workable boxer engine exhaust with collector mounted sensor will likely have a LOT more delay.

~Anything reasonably close will bring an improvement in VEAL performance.
LOL ... I built a spreadsheet.
Calculated volume from exhaust valve to O2 sensor.
Calculated time to fill exhaust volume @ 100% fill for 3 RPM points.
Made a *huge* assumption that VE table is representative of % cylinder fill (95 VE = 95% fill)
Built a table using 95, 70, & 45 load at 1000, 3000, & 6000 RPM and plugged it in ... seems reasonable...

I think I can get closer using the available data. Fuel volume should be a linear function of PW. Combined with AFR that should give intake charge volume. Loading a log into a spreadsheet to filter for RPM and load points should then allow calculation of intake charge volume to plug back into the delay spreadsheet ... "reasonably close" is an low bar to set. :-)

Re: Lambda Delay TunerStudio Log Parser

Posted: Thu Nov 09, 2017 12:08 am
by Piledriver
The first time I did it, I took advantage of a bug in one of the early MS2E sequential firmwares that caused random single cycle fuel or ignition dropouts... it turns out my ancient JAW 1.03 WBO2 responded fast enough to see that even at 6K.
Took a bit of trolling around in the logs but it was frequent enough I got good delay data.

When I set my MS3 up, the delays changed a bit seemingly due to how the MS3 processes the data.
I made some temporary mods to the VE table at 3 RPMs with narrow RPM bins for extra fuel with minimal blend.
That worked too.

The MS3 in-memory table is 12x12 so that could take time to manually populate.
A tool either in the firmware itself or in TS is really needed.

I plugged the 3x3 table I use in VEAL into the 12x12 MS3 table and simply extrapolated everything else.
MS3 can use th 12x12 table for EGO correction, the closer it is to correct, the better it can work.