[bug #51255] Calculations in util/setbaud.h are incorrect

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[bug #51255] Calculations in util/setbaud.h are incorrect

Joerg Wunsch-6
URL:
  <http://savannah.nongnu.org/bugs/?51255>

                 Summary: Calculations in util/setbaud.h are incorrect
                 Project: AVR C Runtime Library
            Submitted by: dense505
            Submitted on: Sat 17 Jun 2017 05:56:37 AM UTC
                Category: Header
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Header files
                  Status: None
        Percent Complete: 0%
             Assigned to: None
        Originator Email:
             Open/Closed: Open
         Discussion Lock: Any
                 Release: 2.0.0
           Fixed Release: None

    _______________________________________________________

Details:

Calculations are wrong, the error becomes severe at high baud rates.  Correct
formula in attached file includes correct calculation and finesse of (ratio *
100 + 50)/100 to achieve correct rounding to nearest integer value, not just
truncation.

This should also address bug #37758 as the issue is more general than just a
problem for Xmega devices.  I personally find the updated formula version
easier to follow than the suggestion in bug #37758.

table of results from attached spreadsheet, looks weird with the limited width
to display here, hope it comes out at your end.  The corrected formula agrees
exactly with the corresponding table in an Atmel datasheet.

Osc 16000000 error % after rounding
Baud rate UBRR bad UBRR error % Rounded UBRR Rounded bad UBRR Correct UBRR bad
UBRR
1200 832.3 833.9 0.19 832 834 0.04 -0.20
2400 415.7 417.2 0.36 416 417 -0.08 -0.32
4800 207.3 208.8 0.72 207 209 0.16 -0.80
9600 103.2 104.7 1.45 103 105 0.16 -1.78
14400 68.4 69.9 2.19 68 70 0.65 -2.27
19200 51.1 52.6 2.94 51 53 0.16 -3.75
28800 33.7 35.2 4.45 34 35 -0.82 -3.79
38400 25.0 26.5 5.99 25 27 0.17 -7.82
57600 16.4 17.9 9.17 16 18 2.21 -10.02
76800 12.0 13.5 12.48 12 14 0.17 -16.46
115200 7.7 9.2 19.53 8 9 -4.16 -17.18
230400 3.3 4.8 44.91 3 5 10.19 -49.69
250000 3.0 4.5 50.00 3 5 0.00 -66.67
500000 1.0 2.5 150.00 1 3 0.00 -200.00




    _______________________________________________________

File Attachments:


-------------------------------------------------------
Date: Sat 17 Jun 2017 05:56:37 AM UTC  Name: setbaud corrected.h  Size: 7kB  
By: dense505

<http://savannah.nongnu.org/bugs/download.php?file_id=40929>
-------------------------------------------------------
Date: Sat 17 Jun 2017 05:56:37 AM UTC  Name: setbuad errors.xlsx  Size: 14kB  
By: dense505

<http://savannah.nongnu.org/bugs/download.php?file_id=40930>

    _______________________________________________________

Reply to this item at:

  <http://savannah.nongnu.org/bugs/?51255>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/


_______________________________________________
AVR-libc-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-libc-dev