far addresses

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

far addresses

Bob von Knobloch-2
Hi,
I'm rewriting a project I originally had running on a ATMega 644.
Now I need more space, so an ATMega 1284 is my choice.
I need to have a very large amount of constant data in flash (more than fits under the 64k address range).
My problem is that I must get the address of some constants in high (above 0x10000) memory and here the 'address of' a char[] does not return the right value.

Example:
I have string constant 'mystring' at address  0x11808 (can see it in the .map and .lst files). Using 'print_func(mystring);', which should print this address, albeit truncated to 16bits, returns '0x1e57'. I cannot see a connection here.
Anyone tried to do similar things?

Regards,

Bob von Knobloch
--
The Sun is out, the sky is blue, it's time to drive the MR2.

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Rolf Pfister-2
I also had an issue with a program bigger than 64K.
But this was on an Atmega2560. The problem was the
programmer didnt write blocks above 64K to the
right place.
Now I also used the Atmega1284P for some smaller
projects. Never used yet more than 64K.
I think your Example is too short to give an idea
where your problem is.

Rolf


Am 25.08.2017 um 11:02 schrieb Bob von Knobloch:

> Hi,
> I'm rewriting a project I originally had running on a ATMega 644.
> Now I need more space, so an ATMega 1284 is my choice.
> I need to have a very large amount of constant data in flash (more than
> fits under the 64k address range).
> My problem is that I must get the address of some constants in high
> (above 0x10000) memory and here the 'address of' a char[] does not
> return the right value.
>
> Example:
> I have string constant 'mystring' at address  0x11808 (can see it in the
> .map and .lst files). Using 'print_func(mystring);', which should print
> this address, albeit truncated to 16bits, returns '0x1e57'. I cannot see
> a connection here.
> Anyone tried to do similar things?
>
> Regards,
>
> Bob von Knobloch


_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Bob von Knobloch-2
On 25/08/17 15:18, Rolf Pfister wrote:
> I also had an issue with a program bigger than 64K.
> But this was on an Atmega2560. The problem was the
> programmer didnt write blocks above 64K to the
> right place.

This is not my problem - all is correctly programmed.

> Now I also used the Atmega1284P for some smaller
> projects. Never used yet more than 64K.
> I think your Example is too short to give an idea
> where your problem is.
>
Well, the problem is that the compiler doesn't appear to return the correct address for constants (even when only the lower 16bits are examined) when accessing objects above 64k.
It must be capable internally, I have code running there.
--
The Sun is out, the sky is blue, it's time to drive the MR2.

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Rolf Pfister-2
Am 25.08.2017 um 15:27 schrieb Bob von Knobloch:
> Well, the problem is that the compiler doesn't appear to return the
> correct address for constants (even when only the lower 16bits are
> examined) when accessing objects above 64k.
> It must be capable internally, I have code running there.

Code example, compiler version?
If you think its a compiler bug, maybe it could be helpful to
look at the created assembler code.

Rolf

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Bob von Knobloch-2
On 25/08/17 16:31, Rolf Pfister wrote:

> Am 25.08.2017 um 15:27 schrieb Bob von Knobloch:
>> Well, the problem is that the compiler doesn't appear to return the correct address for constants (even when only the lower 16bits are examined) when accessing objects above 64k.
>> It must be capable internally, I have code running there.
>
> Code example, compiler version?
> If you think its a compiler bug, maybe it could be helpful to
> look at the created assembler code.
>
> Rolf
>
> _______________________________________________
> AVR-chat mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/avr-chat
>
Hi Rolf,

Compiler is:
avr-gcc (AVR_8_bit_GNU_Toolchain_3.5.0_1662) 4.9.2

code is like:

const char teststring[] PROGMEM = "some text"; // Loaded with a SECTION statement and a linker map at 0x01b000 (high memory) (and checked that it really is there)

hex32out(teststring); // Prints the hex value of the address of teststring

result: 00007187 // No relation to the address

Anyway, Jörg Wünsch has commented somewhere that all such strings should be put into low memory, so I think this is an indication that the compiler cannot be used this way. I wouldn't call it a bug as the compiler is constructed for processors with a 16-bit address space. Just that AVR have extended it by using special instructions that we probably can't expect the compiler to support.
I have reorganised my code so that all symbolically addressed objects reside under 0x10000 and those above are numerically addressed (luckily I could do this, originally I had otherwise planned).

Anyway thanks for replying and maybe this helps someone.

Cheers,
Bob

--
The Sun is out, the sky is blue, it's time to drive the MR2.

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Rogier Wolff
On Tue, Aug 29, 2017 at 11:54:24AM +0200, Bob von Knobloch wrote:

> I wouldn't call it a bug as the compiler is constructed for
> processors with a 16-bit address space.

Nononono! no! The compiler (gcc) is constructed for devices with a
32-bit address space. It has been quite reasonably hacked to work for
the 16-bit address space that the AVR has....

But I remember the days when my co-students were swearing at gcc
because it was not easy to get it to cooperate with a 16-bit address
space.... (and there are bugs you are way more likely to encounter
in a 64 k address space than in 4G. For example, what happens
when you have:
   print_bytes (char *p, int len)
   {
     char *q;
     for (q=p+len;p < q;p++)
        putchar (*p);
   }

Works, right? Now have p point at just below halfway your addressing
space (say 0x7ff0) and q after (say 0x8010).... IIRC the above is
gcc's own idea for an optimization when you actually write
      for (i=0;i<len;i++,p++)
        putchar (*p);
Has this bug been fixed? It's been about 25 years since I reported
it....)

For AVR there is a big penalty when you are say indexing an array
where both the base address and the index are 16-bit numbers as
opposed to one of them only 8-bit.

You might use a trick: just store the upper 16 bits, the bottom bit is
always zero. This way you can have pointers across the whole 17-bit
progrmem. But you expect things like:

  print_nibble (uint8_t b)
  {
    char digits="0123456789abcdef";
    putchar (digits[b]);
  }

to work. Or stuff where you walk a string one byte at a time:
  print_string (/*progmem*/char *p)
  {
  while (*p)
    putchar (*p++);
  }

So to make stuff like that work, the only opption is to use 24-bit
progmem pointers. Again a big penalty for MOST operations.

        Roger.

--
** [hidden email] ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
The plan was simple, like my brother-in-law Phil. But unlike
Phil, this plan just might work.

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Rolf Pfister-2
In reply to this post by Bob von Knobloch-2
Am 29.08.2017 um 11:54 schrieb Bob von Knobloch:
> Compiler is:
> avr-gcc (AVR_8_bit_GNU_Toolchain_3.5.0_1662) 4.9.2
>

I have the same version and tried a bit today.


> code is like:
>
> const char teststring[] PROGMEM = "some text";         // Loaded with a
> SECTION statement and a linker map at 0x01b000 (high memory) (and
> checked that it really is there)
>
> hex32out(teststring);                    // Prints the hex value of the
> address of teststring
>
> result: 00007187                    // No relation to the address
>

I couldn't reproduce this behavior.
When anding with 0xFFFF this part of the address was always correct.
The problem is the above bit. When anding with 0x1FFFF I get wrong
addresses. Sometimes even low addresses will have the upper bit set.

So my solution is to set the bit when I know to have a string above
64KB. Otherwise anding with 0xFFFF.
So my example program looks like this:

#include <avr/pgmspace.h>
typedef char String100[100];
const String100 mystrings[3] PROGMEM = {
  "mystrings 12345...",
  "123456789 12345...",
  "Last string...",
};

void print_addr_progmem(const char *b0,bool far=false)
{
  unsigned long addr = ((long)b0) & 0x0FFFF;
  if(far)  addr |= 0x10000; //should work above 64KB for Atmega1284P
  zahl_anzeigen(addr,ROT,16); //print address of string (16=Hex)
  char txt[16];
  int i;
  for(i=0;i<10;i++)
   {
    txt[i] = __ELPM_classic__(addr++);
    if(txt[i]==0) break;
   }
  txt[i]=0;
  led_print(txt); //should print the string (first 10 characters)
}

void progmem_test()
{
  //above 64KB:
  print_addr_progmem(mystrings[0],true);
  //below 64KB:
  print_addr_progmem(manystrings[0]);
}

I had to create quite a lot of strings to get above the 64KB limit.
When having not too many the compiler put them at lower addresses
and program code will go to higher addresses. So most the time
it will not be a problem.

Rolf

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Bob Paddock-3
In reply to this post by Bob von Knobloch-2
Not sure if your issue is the related to this one?:

https://lists.nongnu.org/archive/html/avr-libc-dev/2010-09/msg00046.html

As Bob Paddock wrote:

> From the error that I'm getting, below, I belive that __data_load_end
> is only 16 bits wide?

No, obviously it's wider:

 0x0003c370   __data_load_start = LOADADDR (.data)
 0x0003c378   __data_load_end = (__data_load_start + SIZEOF (.data))

(That's from a map file.)

Note that the linker has no notion of what C knows as a "type".  It
only has symbols.

>   uint16_t flash_end_u16 = (uint16_t) &(__data_load_end[0]);

Your problem is that the & operator makes a pointer out of something
else, and pointers in AVR-GCC are fixed to 16 bits in size.  You have
to avoid turning the large integer __data_load_end actually is into a
pointer (syntactically).  I couldn't find any way using C syntax for
this, unless resorting straight to assembly:

static inline uint32_t get_data_load_end(void)
{
  uint32_t tmp;

  asm volatile("ldi %A0, lo8(__data_load_end)" "\n"
               "ldi %B0, hi8(__data_load_end)" "\n"
               "ldi %C0, hlo8(__data_load_end)" "\n"
               "ldi %D0, hhi8(__data_load_end)"
               : "=r"(tmp));
  return tmp;
}



On Fri, Aug 25, 2017 at 5:02 AM, Bob von Knobloch <[hidden email]> wrote:

> Hi,
> I'm rewriting a project I originally had running on a ATMega 644.
> Now I need more space, so an ATMega 1284 is my choice.
> I need to have a very large amount of constant data in flash (more than fits
> under the 64k address range).
> My problem is that I must get the address of some constants in high (above
> 0x10000) memory and here the 'address of' a char[] does not return the right
> value.
>
> Example:
> I have string constant 'mystring' at address  0x11808 (can see it in the
> .map and .lst files). Using 'print_func(mystring);', which should print this
> address, albeit truncated to 16bits, returns '0x1e57'. I cannot see a
> connection here.
> Anyone tried to do similar things?
>
> Regards,
>
> Bob von Knobloch
> --
> The Sun is out, the sky is blue, it's time to drive the MR2.
>
> _______________________________________________
> AVR-chat mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/avr-chat

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Bill Gatliff
Given that what you're doing is highly HIGHLY representation-specific, assembly language is probably the most direct and reliable approach. There might be a gcc __attribute__ hack to help, but then you'd be just trading assembly language for something compiler-specific AND representation-specific.

It's inevitable that when data spills over into a memory space with a different access schema, a C compiler goes nuts: the machine abstraction that C is defined around supports no such concept. That's doubly so for a general-purpose compiler like gcc, where exceptions to those rules all have to be dealt with outside the compiler's main workflow.

b.g.


On Thu, Aug 31, 2017 at 1:14 PM Bob Paddock <[hidden email]> wrote:
Not sure if your issue is the related to this one?:

https://lists.nongnu.org/archive/html/avr-libc-dev/2010-09/msg00046.html

As Bob Paddock wrote:

> From the error that I'm getting, below, I belive that __data_load_end
> is only 16 bits wide?

No, obviously it's wider:

 0x0003c370   __data_load_start = LOADADDR (.data)
 0x0003c378   __data_load_end = (__data_load_start + SIZEOF (.data))

(That's from a map file.)

Note that the linker has no notion of what C knows as a "type".  It
only has symbols.

>   uint16_t flash_end_u16 = (uint16_t) &(__data_load_end[0]);

Your problem is that the & operator makes a pointer out of something
else, and pointers in AVR-GCC are fixed to 16 bits in size.  You have
to avoid turning the large integer __data_load_end actually is into a
pointer (syntactically).  I couldn't find any way using C syntax for
this, unless resorting straight to assembly:

static inline uint32_t get_data_load_end(void)
{
  uint32_t tmp;

  asm volatile("ldi %A0, lo8(__data_load_end)" "\n"
               "ldi %B0, hi8(__data_load_end)" "\n"
               "ldi %C0, hlo8(__data_load_end)" "\n"
               "ldi %D0, hhi8(__data_load_end)"
               : "=r"(tmp));
  return tmp;
}



On Fri, Aug 25, 2017 at 5:02 AM, Bob von Knobloch <[hidden email]> wrote:
> Hi,
> I'm rewriting a project I originally had running on a ATMega 644.
> Now I need more space, so an ATMega 1284 is my choice.
> I need to have a very large amount of constant data in flash (more than fits
> under the 64k address range).
> My problem is that I must get the address of some constants in high (above
> 0x10000) memory and here the 'address of' a char[] does not return the right
> value.
>
> Example:
> I have string constant 'mystring' at address  0x11808 (can see it in the
> .map and .lst files). Using 'print_func(mystring);', which should print this
> address, albeit truncated to 16bits, returns '0x1e57'. I cannot see a
> connection here.
> Anyone tried to do similar things?
>
> Regards,
>
> Bob von Knobloch
> --
> The Sun is out, the sky is blue, it's time to drive the MR2.
>
> _______________________________________________
> AVR-chat mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/avr-chat

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
--

Bill Gatliff
Embedded Linux training and consulting
[hidden email]
(309) 453-3421


_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Bob von Knobloch-2
On 31/08/17 20:26, Bill Gatliff wrote:

>
> It's inevitable that when data spills over into a memory space with a
> different access schema, a C compiler goes nuts: the machine abstraction
> that C is defined around supports no such concept. That's doubly so for a
> general-purpose compiler like gcc, where exceptions to those rules all have
> to be dealt with outside the compiler's main workflow.
>
>
Yes, that is the conclusion I had also come to.
I see that Bob Paddock's reply (that avr-gcc turns the output of '&' into a 16 bit pointer) explains the limitation (but not the 'mangling' that I see).
And Rolf Pfister
"I had to create quite a lot of strings to get above the 64KB limit.
When having not too many the compiler put them at lower addresses
and program code will go to higher addresses. So most the time
it will not be a problem."

My system, however, set these specifically in high memory using a linker map. So this was a real problem.
Reorganisation has mitigated the problem for now.
Thanks to all who contributed.

Bob

--
The Sun is out, the sky is blue, it's time to drive the MR2.

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat
Reply | Threaded
Open this post in threaded view
|

Re: far addresses

Michael Hennebry
In reply to this post by Bob von Knobloch-2
I think avr-gcc 4.9 has named address spaces.
The __memx qualifier will do what you want.
Pointers to __memx are 24 bits.
The __memx namespace is the union of all available namespaces.
That includes all of flash and SRAM.
So far as I know,
there is no other namespace that includes all available flash.
All the flash-only namespaces are at most 64K each.
With C syntax, you cannot have a single data structure with more than 64K.
However you make it, I think you can still use its name:
extern const __memx type fred[];

--
Michael   [hidden email]
"Sorry but your password must contain an uppercase letter, a number,
a haiku, a gang sign, a heiroglyph, and the blood of a virgin."
                                                              --  someeecards

_______________________________________________
AVR-chat mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/avr-chat