librelist archives

« back to archive

Exception: hashindex_read failed

Exception: hashindex_read failed

From:
Evan Hempel
Date:
2015-02-07 @ 03:05
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I'm getting this exception on Attic 0.14 and the latest Attic from
github.  The repository is somewhat over 600Gb with 21 archives inside.

Full exception:

root@krakatoa:/backup/bind# time attic create --exclude-from
/etc/atticmatic/excludes --verbose --stats
/backup/attic/repository::krakatoa-2015-01-04T00:20:26.546022 /backup/bind
hashindex:

/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks:
fread failed
Traceback (most recent call last):
  File "/usr/local/bin/attic", line 3, in <module>
    main()
  File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
line 727, in main
    exit_code = archiver.run(sys.argv[1:])
  File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
line 717, in run
    return args.func(args)
  File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
line 100, in do_create
    cache = Cache(repository, key, manifest)
  File "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line
28, in __init__
    self.open()
  File "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line
68, in open
    self.chunks = ChunkIndex.read(os.path.join(self.path,
'chunks').encode('utf-8'))
  File "hashindex.pyx", line 47, in attic.hashindex.IndexBase.read
(attic/hashindex.c:1209)
  File "hashindex.pyx", line 35, in
attic.hashindex.IndexBase.__cinit__ (attic/hashindex.c:1037)
Exception: hashindex_read failed

real    0m9.755s
user    0m0.491s
sys     0m2.478s


root@krakatoa:/backup/bind# ls -l

/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
- -rw-r--r-- 1 root root 738197522 Jan  4 01:09

/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks

root@krakatoa:/backup/bind# xxd

/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
| head -n 20
0000000: 4154 5449 4349 4458 a9c0 c000 0000 0001  ATTICIDX........
0000010: 200c 0000 0039 6fa0 c8fe 1119 12ca 3d88   ....9o.......=.
0000020: 4d5c 9183 4d6e cf6a 2305 329a 9bc7 db49  M\..Mn.j#.2....I
0000030: 8e07 1500 0000 3f5c 0100 8d4f 0000 0100  ......?\...O....
0000040: 0009 6bcd 2ec4 9392 fc35 bb33 27df 1325  ..k......5.3'..%
0000050: 2354 d58f e03e d039 21af c7ef d9e4 1500  #T...>.9!.......
0000060: 0000 de77 0000 ef77 0000 0200 0027 942c  ...w...w.....'.,
0000070: 54a5 920c 6c68 882d aca7 3f88 843f 47d8  T...lh.-..?..?G.
0000080: 7f59 b9d5 c60d 356d 7bdb 1500 0000 2dee  .Y....5m{.....-.
0000090: 0000 12d6 0000 0300 0001 851f e2ad 53c5  ..............S.
00000a0: 9b10 c454 31b5 1bb0 e6d8 b112 7f38 d025  ...T1........8.%
00000b0: db1b 2201 31f7 1500 0000 e03f 0000 ec3f  ..".1......?...?
00000c0: 0000 0200 0028 cbce bea6 9118 9ee5 d44d  .....(.........M
00000d0: e113 c26c e7bf 8ca9 8367 8f66 e4a6 8915  ...l.....g.f....
00000e0: 9bb8 1400 0000 6821 0000 6421 0000 fcff  ......h!..d!....
00000f0: fff6 f1f9 4b8f 6f4b b09e 750e 2c9c 497e  ....K.oK..u.,.I~
0000100: 06d3 e7ca e227 5c23 8a62 9b71 5f2b 2000  .....'\#.b.q_+ .
0000110: 0000 630c 0400 3140 0200 0600 00fc 02a2  ..c...1@........
0000120: eaa9 408b 22d3 5e7d 5c60 a8ac 503d 1e7a  ..@.".^}\`..P=.z
0000130: 7755 322c 5cf5 46ff 169d 1500 0000 a2ff  wU2,\.F.........


I hit this first using Attic 0.14, but then read the "Does attic have
an upper data size limit?" thread and since the exception there looks
similar I uninstall 0.14 then installed the latest version from git,
and I'm still seeing the same issue.

I've dug into the code enough to see that the error is on a different
line than the fix in the other thread (attic/_hashindex.c line 165 vs
line 180).

I'm not sure where I should go from here ...

Evan Hempel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJU1YDdAAoJEBQYbQX1fj+hnnMQAIq8nJ4Kn6ZzX+T2A0kSG3Ns
L537NjgBYNRdMOyhDkPvI6UVnF+X6NcdBTg9iu6tVMrRNBVql0SdA+7oSKehOMow
KYlBFTnt7BnbJxGOYfi3zdu6VqpDMWdnFQwcObfBppFT22ScuRDpD8OibN11PYZ/
Us+9TlX3zo85Bl+3OPwTiq47saNDew/8J9XcWuqMwCu7X3D6oAoWIWTxQbQMl1qZ
RPxzxJgT3LjkA53gYL0MbGB20MxWKEU95FrX8haK5y6djpO+0w41ihlmECk7bPuh
mWQEyo5Rs7OsRK41fmA+9HpFFHuZgMPpdUwn7ELxUzcAC5N/PylOsH5g00zNEQ6b
1dXbZiyD1gKizGclPdVpt4vEgEvR7GpnJ22M9kglYef5u4aAhQicV2xCTG70c7gv
ZgVrIOg+heeEi69uy4OUGHhc4PyZertslpXARZbg4dED3XudB4mvU25UdXtkUF9L
gODsetA25m9cgYNblbK4T4nBM9ziXlqGZdWn6claQXZ1fmFQRl/SnfRCf5WRnlCA
AG40cT3BUZvO7UV0empAGsBwJtDtJKNq40203L/16T5WFxxRIKgwqyvkB/Qe37Tf
J6/N5Iio+fJvVwRY/2DA1DGurViKqxvuPXrc7/JfJXgWkxGysIF5wHA3q5+D1FI3
4aZmHh2CmFIoOXn/U6vF
=r3fX
-----END PGP SIGNATURE-----

Re: [attic] Exception: hashindex_read failed [SOLVED]

From:
Evan Hempel
Date:
2015-02-13 @ 01:47
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Update...

I modified[1] attic/_hashindex.c to print out the expected read size
vs what it actually ended up reading.  Turns out it was expecting
738197522 bytes but is only reading 279478272 bytes (that number is
stable, I get the same result on multiple runs even after dropping
disk caches).

root@krakatoa:/backup/bind# time attic create --exclude-from
/etc/atticmatic/excludes --verbose --stats
/backup/attic/repository::krakatoa-2015-01-04T00:20:26.546022 /backup/bind
hashindex:

/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks:
fread hashindex failed (expected 738197522, got 279478272)
Traceback (most recent call last):
...

So ... guessing there might be a disk error I tried dd'ing the file to
/dev/null, and lo and behold, a read error 279 Mb in.

root@krakatoa:/backup/bind# time dd

if=/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
of=/dev/null
dd: error reading

‘/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks’:
Input/output error
545856+0 records in
545856+0 records out
279478272 bytes (279 MB) copied, 4.15514 s, 67.3 MB/s


Time for me to get a new disk :-/

Evan


[1]

https://github.com/evanhempel/attic/commit/b425950f8b13f48bda2fb3672a36737e801b0e06


On 02/06/2015 10:05 PM, Evan Hempel wrote:
> Hi,
> 
> I'm getting this exception on Attic 0.14 and the latest Attic from 
> github.  The repository is somewhat over 600Gb with 21 archives
> inside.
> 
> Full exception:
> 
> root@krakatoa:/backup/bind# time attic create --exclude-from 
> /etc/atticmatic/excludes --verbose --stats 
> /backup/attic/repository::krakatoa-2015-01-04T00:20:26.546022
> /backup/bind hashindex: 
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks:
>
> 
fread failed
> Traceback (most recent call last): File "/usr/local/bin/attic",
> line 3, in <module> main() File
> "/usr/local/lib/python3.4/dist-packages/attic/archiver.py", line
> 727, in main exit_code = archiver.run(sys.argv[1:]) File
> "/usr/local/lib/python3.4/dist-packages/attic/archiver.py", line
> 717, in run return args.func(args) File
> "/usr/local/lib/python3.4/dist-packages/attic/archiver.py", line
> 100, in do_create cache = Cache(repository, key, manifest) File
> "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line 28,
> in __init__ self.open() File
> "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line 68,
> in open self.chunks = ChunkIndex.read(os.path.join(self.path, 
> 'chunks').encode('utf-8')) File "hashindex.pyx", line 47, in
> attic.hashindex.IndexBase.read (attic/hashindex.c:1209) File
> "hashindex.pyx", line 35, in attic.hashindex.IndexBase.__cinit__
> (attic/hashindex.c:1037) Exception: hashindex_read failed
> 
> real    0m9.755s user    0m0.491s sys     0m2.478s
> 
> 
> root@krakatoa:/backup/bind# ls -l 
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
>
> 
- -rw-r--r-- 1 root root 738197522 Jan  4 01:09
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
>
>  root@krakatoa:/backup/bind# xxd 
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
>
> 
| head -n 20
> 0000000: 4154 5449 4349 4458 a9c0 c000 0000 0001  ATTICIDX........ 
> 0000010: 200c 0000 0039 6fa0 c8fe 1119 12ca 3d88   ....9o.......=. 
> 0000020: 4d5c 9183 4d6e cf6a 2305 329a 9bc7 db49  M\..Mn.j#.2....I 
> 0000030: 8e07 1500 0000 3f5c 0100 8d4f 0000 0100  ......?\...O.... 
> 0000040: 0009 6bcd 2ec4 9392 fc35 bb33 27df 1325  ..k......5.3'..% 
> 0000050: 2354 d58f e03e d039 21af c7ef d9e4 1500  #T...>.9!....... 
> 0000060: 0000 de77 0000 ef77 0000 0200 0027 942c  ...w...w.....'., 
> 0000070: 54a5 920c 6c68 882d aca7 3f88 843f 47d8  T...lh.-..?..?G. 
> 0000080: 7f59 b9d5 c60d 356d 7bdb 1500 0000 2dee  .Y....5m{.....-. 
> 0000090: 0000 12d6 0000 0300 0001 851f e2ad 53c5  ..............S. 
> 00000a0: 9b10 c454 31b5 1bb0 e6d8 b112 7f38 d025  ...T1........8.% 
> 00000b0: db1b 2201 31f7 1500 0000 e03f 0000 ec3f  ..".1......?...? 
> 00000c0: 0000 0200 0028 cbce bea6 9118 9ee5 d44d  .....(.........M 
> 00000d0: e113 c26c e7bf 8ca9 8367 8f66 e4a6 8915  ...l.....g.f.... 
> 00000e0: 9bb8 1400 0000 6821 0000 6421 0000 fcff  ......h!..d!.... 
> 00000f0: fff6 f1f9 4b8f 6f4b b09e 750e 2c9c 497e  ....K.oK..u.,.I~ 
> 0000100: 06d3 e7ca e227 5c23 8a62 9b71 5f2b 2000  .....'\#.b.q_+ . 
> 0000110: 0000 630c 0400 3140 0200 0600 00fc 02a2  ..c...1@........ 
> 0000120: eaa9 408b 22d3 5e7d 5c60 a8ac 503d 1e7a  ..@.".^}\`..P=.z 
> 0000130: 7755 322c 5cf5 46ff 169d 1500 0000 a2ff  wU2,\.F.........
> 
> 
> I hit this first using Attic 0.14, but then read the "Does attic
> have an upper data size limit?" thread and since the exception
> there looks similar I uninstall 0.14 then installed the latest
> version from git, and I'm still seeing the same issue.
> 
> I've dug into the code enough to see that the error is on a
> different line than the fix in the other thread (attic/_hashindex.c
> line 165 vs line 180).
> 
> I'm not sure where I should go from here ...
> 
> Evan Hempel
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJU3Ve+AAoJEBQYbQX1fj+hEdwP/3yjb7WWQ5gb4RUzmgNPuaJV
CbLupfDfefiugh3+tkNS3nq7LqJNXk4hc5XSiFmiHF0OrvnDocN4qvbYTt2ASL/b
kpbSAkLN+3cpiagFkDI+G2RuUz0DSmKF1jJdW25rrSAaGnr6zP9ayJhTak2lqtgZ
F6wEEBvxe0d9uhDw0fl42pGbdK4sO0AiSlPFRncCKlPQ+/DQJvbZoJk7ho6cWMnM
bRbsVKJ5Ir5AJYj7q2ae9dzeMWI3fGLq0+CWBM6hJQjePAPiruS2YFjyuRyuV+Nm
mptP87lT5HEHaCUkIh8IMn9HginWibdwC9l+9zDVSmxbM5XyXvjAPzlHDkzz6Qxd
4zuMJkptucxqHkRZvvZk3ALa/wJBQINv4v14i36UO6tcKmBIC/n/7MTC+TxF6BnG
9J+He/jPZwfs70kMM+KMOLa4ldjcCR7SdyLu/4ypXzjBrAT2kAlFeg/bdPlwXISf
UUS08AKmMuxS9MwY8sO32REVpxKIgWhQ5zKuEHcoM06z+Qq5NHP3lvFyHIXWfYeK
p1iCN1I62sneETPQJIGTe4BLSk6DLfIUtdf5YfbzgrSRSHtDGdbwWm1OdUjRQTyR
WnjlmXMBdQaJn90IAfvxYhqW7kUQPKr1MZDxBZ00jNSWEWxkchzyqOW9qlvh9H3j
rr6JA3WEeEB7Cpxu9ZLK
=dvhv
-----END PGP SIGNATURE-----

Re: [attic] Exception: hashindex_read failed

From:
Thiago Coutinho
Date:
2015-02-11 @ 19:30
I'm having this same issue.

@Evan: do you have found a fix?

2015-02-07 1:05 GMT-02:00 Evan Hempel <evanhempel@evanhempel.com>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I'm getting this exception on Attic 0.14 and the latest Attic from
> github.  The repository is somewhat over 600Gb with 21 archives inside.
>
> Full exception:
>
> root@krakatoa:/backup/bind# time attic create --exclude-from
> /etc/atticmatic/excludes --verbose --stats
> /backup/attic/repository::krakatoa-2015-01-04T00:20:26.546022 /backup/bind
> hashindex:
>
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks:
> fread failed
> Traceback (most recent call last):
>   File "/usr/local/bin/attic", line 3, in <module>
>     main()
>   File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
> line 727, in main
>     exit_code = archiver.run(sys.argv[1:])
>   File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
> line 717, in run
>     return args.func(args)
>   File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
> line 100, in do_create
>     cache = Cache(repository, key, manifest)
>   File "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line
> 28, in __init__
>     self.open()
>   File "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line
> 68, in open
>     self.chunks = ChunkIndex.read(os.path.join(self.path,
> 'chunks').encode('utf-8'))
>   File "hashindex.pyx", line 47, in attic.hashindex.IndexBase.read
> (attic/hashindex.c:1209)
>   File "hashindex.pyx", line 35, in
> attic.hashindex.IndexBase.__cinit__ (attic/hashindex.c:1037)
> Exception: hashindex_read failed
>
> real    0m9.755s
> user    0m0.491s
> sys     0m2.478s
>
>
> root@krakatoa:/backup/bind# ls -l
>
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
> - -rw-r--r-- 1 root root 738197522 Jan  4 01:09
>
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
>
> root@krakatoa:/backup/bind# xxd
>
> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
> | head -n 20
> 0000000: 4154 5449 4349 4458 a9c0 c000 0000 0001  ATTICIDX........
> 0000010: 200c 0000 0039 6fa0 c8fe 1119 12ca 3d88   ....9o.......=.
> 0000020: 4d5c 9183 4d6e cf6a 2305 329a 9bc7 db49  M\..Mn.j#.2....I
> 0000030: 8e07 1500 0000 3f5c 0100 8d4f 0000 0100  ......?\...O....
> 0000040: 0009 6bcd 2ec4 9392 fc35 bb33 27df 1325  ..k......5.3'..%
> 0000050: 2354 d58f e03e d039 21af c7ef d9e4 1500  #T...>.9!.......
> 0000060: 0000 de77 0000 ef77 0000 0200 0027 942c  ...w...w.....'.,
> 0000070: 54a5 920c 6c68 882d aca7 3f88 843f 47d8  T...lh.-..?..?G.
> 0000080: 7f59 b9d5 c60d 356d 7bdb 1500 0000 2dee  .Y....5m{.....-.
> 0000090: 0000 12d6 0000 0300 0001 851f e2ad 53c5  ..............S.
> 00000a0: 9b10 c454 31b5 1bb0 e6d8 b112 7f38 d025  ...T1........8.%
> 00000b0: db1b 2201 31f7 1500 0000 e03f 0000 ec3f  ..".1......?...?
> 00000c0: 0000 0200 0028 cbce bea6 9118 9ee5 d44d  .....(.........M
> 00000d0: e113 c26c e7bf 8ca9 8367 8f66 e4a6 8915  ...l.....g.f....
> 00000e0: 9bb8 1400 0000 6821 0000 6421 0000 fcff  ......h!..d!....
> 00000f0: fff6 f1f9 4b8f 6f4b b09e 750e 2c9c 497e  ....K.oK..u.,.I~
> 0000100: 06d3 e7ca e227 5c23 8a62 9b71 5f2b 2000  .....'\#.b.q_+ .
> 0000110: 0000 630c 0400 3140 0200 0600 00fc 02a2  ..c...1@........
> 0000120: eaa9 408b 22d3 5e7d 5c60 a8ac 503d 1e7a  ..@.".^}\`..P=.z
> 0000130: 7755 322c 5cf5 46ff 169d 1500 0000 a2ff  wU2,\.F.........
>
>
> I hit this first using Attic 0.14, but then read the "Does attic have
> an upper data size limit?" thread and since the exception there looks
> similar I uninstall 0.14 then installed the latest version from git,
> and I'm still seeing the same issue.
>
> I've dug into the code enough to see that the error is on a different
> line than the fix in the other thread (attic/_hashindex.c line 165 vs
> line 180).
>
> I'm not sure where I should go from here ...
>
> Evan Hempel
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
>
> iQIcBAEBAgAGBQJU1YDdAAoJEBQYbQX1fj+hnnMQAIq8nJ4Kn6ZzX+T2A0kSG3Ns
> L537NjgBYNRdMOyhDkPvI6UVnF+X6NcdBTg9iu6tVMrRNBVql0SdA+7oSKehOMow
> KYlBFTnt7BnbJxGOYfi3zdu6VqpDMWdnFQwcObfBppFT22ScuRDpD8OibN11PYZ/
> Us+9TlX3zo85Bl+3OPwTiq47saNDew/8J9XcWuqMwCu7X3D6oAoWIWTxQbQMl1qZ
> RPxzxJgT3LjkA53gYL0MbGB20MxWKEU95FrX8haK5y6djpO+0w41ihlmECk7bPuh
> mWQEyo5Rs7OsRK41fmA+9HpFFHuZgMPpdUwn7ELxUzcAC5N/PylOsH5g00zNEQ6b
> 1dXbZiyD1gKizGclPdVpt4vEgEvR7GpnJ22M9kglYef5u4aAhQicV2xCTG70c7gv
> ZgVrIOg+heeEi69uy4OUGHhc4PyZertslpXARZbg4dED3XudB4mvU25UdXtkUF9L
> gODsetA25m9cgYNblbK4T4nBM9ziXlqGZdWn6claQXZ1fmFQRl/SnfRCf5WRnlCA
> AG40cT3BUZvO7UV0empAGsBwJtDtJKNq40203L/16T5WFxxRIKgwqyvkB/Qe37Tf
> J6/N5Iio+fJvVwRY/2DA1DGurViKqxvuPXrc7/JfJXgWkxGysIF5wHA3q5+D1FI3
> 4aZmHh2CmFIoOXn/U6vF
> =r3fX
> -----END PGP SIGNATURE-----
>



-- 
Thiago Coutinho

"O povo não deveria temer o governo. O governo é quem deveria temer o povo."
V de Vingança

Re: [attic] Exception: hashindex_read failed

From:
Thiago Coutinho
Date:
2015-02-11 @ 23:25
Actually, my problem is another one:

hashindex: /home/thiagoc/.cache/attic/
614e6b26248d5402482d1df4b21b1bcb93114ff89f7a128e683b7fae67f26074/chunks:
msync failed
Traceback (most recent call last):
  File "/usr/bin/attic", line 3, in <module>
    main()
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 715, in main
    exit_code = archiver.run(sys.argv[1:])
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 705, in run
    return args.func(args)
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 128, in
do_create
    self._process(archive, cache, args.excludes, args.exclude_caches,
skip_inodes, path, restrict_dev)
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 177, in
_process
    os.path.join(path, filename), restrict_dev)
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 177, in
_process
    os.path.join(path, filename), restrict_dev)
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 177, in
_process
    os.path.join(path, filename), restrict_dev)
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 177, in
_process
    os.path.join(path, filename), restrict_dev)
  File "/usr/lib/python3/dist-packages/attic/archiver.py", line 163, in
_process
    archive.process_file(path, st, cache)
  File "/usr/lib/python3/dist-packages/attic/archive.py", line 407, in
process_file
    self.add_item(item)
  File "/usr/lib/python3/dist-packages/attic/archive.py", line 170, in
add_item
    self.write_checkpoint()
  File "/usr/lib/python3/dist-packages/attic/archive.py", line 174, in
write_checkpoint
    self.save(self.checkpoint_name)
  File "/usr/lib/python3/dist-packages/attic/archive.py", line 198, in save
    self.cache.commit()
  File "/usr/lib/python3/dist-packages/attic/cache.py", line 116, in commit
    self.chunks.flush()
  File "hashindex.pyx", line 56, in attic.hashindex.IndexBase.flush
(attic/hashindex.c:1479)
Exception: hashindex_flush failed

I''m still using 0.13. Will update e try again, if that fail I will file a
bug report.


2015-02-11 17:30 GMT-02:00 Thiago Coutinho <root@thiagoc.net>:

> I'm having this same issue.
>
> @Evan: do you have found a fix?
>
> 2015-02-07 1:05 GMT-02:00 Evan Hempel <evanhempel@evanhempel.com>:
>
> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi,
>>
>> I'm getting this exception on Attic 0.14 and the latest Attic from
>> github.  The repository is somewhat over 600Gb with 21 archives inside.
>>
>> Full exception:
>>
>> root@krakatoa:/backup/bind# time attic create --exclude-from
>> /etc/atticmatic/excludes --verbose --stats
>> /backup/attic/repository::krakatoa-2015-01-04T00:20:26.546022 /backup/bind
>> hashindex:
>>
>> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks:
>> fread failed
>> Traceback (most recent call last):
>>   File "/usr/local/bin/attic", line 3, in <module>
>>     main()
>>   File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
>> line 727, in main
>>     exit_code = archiver.run(sys.argv[1:])
>>   File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
>> line 717, in run
>>     return args.func(args)
>>   File "/usr/local/lib/python3.4/dist-packages/attic/archiver.py",
>> line 100, in do_create
>>     cache = Cache(repository, key, manifest)
>>   File "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line
>> 28, in __init__
>>     self.open()
>>   File "/usr/local/lib/python3.4/dist-packages/attic/cache.py", line
>> 68, in open
>>     self.chunks = ChunkIndex.read(os.path.join(self.path,
>> 'chunks').encode('utf-8'))
>>   File "hashindex.pyx", line 47, in attic.hashindex.IndexBase.read
>> (attic/hashindex.c:1209)
>>   File "hashindex.pyx", line 35, in
>> attic.hashindex.IndexBase.__cinit__ (attic/hashindex.c:1037)
>> Exception: hashindex_read failed
>>
>> real    0m9.755s
>> user    0m0.491s
>> sys     0m2.478s
>>
>>
>> root@krakatoa:/backup/bind# ls -l
>>
>> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
>> - -rw-r--r-- 1 root root 738197522 Jan  4 01:09
>>
>> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
>>
>> root@krakatoa:/backup/bind# xxd
>>
>> 
/root/.cache/attic/af80d9706cd115e534574d0c63ba0823b337e01f9dd823ffefd62e99e80ca27d/chunks
>> | head -n 20
>> 0000000: 4154 5449 4349 4458 a9c0 c000 0000 0001  ATTICIDX........
>> 0000010: 200c 0000 0039 6fa0 c8fe 1119 12ca 3d88   ....9o.......=.
>> 0000020: 4d5c 9183 4d6e cf6a 2305 329a 9bc7 db49  M\..Mn.j#.2....I
>> 0000030: 8e07 1500 0000 3f5c 0100 8d4f 0000 0100  ......?\...O....
>> 0000040: 0009 6bcd 2ec4 9392 fc35 bb33 27df 1325  ..k......5.3'..%
>> 0000050: 2354 d58f e03e d039 21af c7ef d9e4 1500  #T...>.9!.......
>> 0000060: 0000 de77 0000 ef77 0000 0200 0027 942c  ...w...w.....'.,
>> 0000070: 54a5 920c 6c68 882d aca7 3f88 843f 47d8  T...lh.-..?..?G.
>> 0000080: 7f59 b9d5 c60d 356d 7bdb 1500 0000 2dee  .Y....5m{.....-.
>> 0000090: 0000 12d6 0000 0300 0001 851f e2ad 53c5  ..............S.
>> 00000a0: 9b10 c454 31b5 1bb0 e6d8 b112 7f38 d025  ...T1........8.%
>> 00000b0: db1b 2201 31f7 1500 0000 e03f 0000 ec3f  ..".1......?...?
>> 00000c0: 0000 0200 0028 cbce bea6 9118 9ee5 d44d  .....(.........M
>> 00000d0: e113 c26c e7bf 8ca9 8367 8f66 e4a6 8915  ...l.....g.f....
>> 00000e0: 9bb8 1400 0000 6821 0000 6421 0000 fcff  ......h!..d!....
>> 00000f0: fff6 f1f9 4b8f 6f4b b09e 750e 2c9c 497e  ....K.oK..u.,.I~
>> 0000100: 06d3 e7ca e227 5c23 8a62 9b71 5f2b 2000  .....'\#.b.q_+ .
>> 0000110: 0000 630c 0400 3140 0200 0600 00fc 02a2  ..c...1@........
>> 0000120: eaa9 408b 22d3 5e7d 5c60 a8ac 503d 1e7a  ..@.".^}\`..P=.z
>> 0000130: 7755 322c 5cf5 46ff 169d 1500 0000 a2ff  wU2,\.F.........
>>
>>
>> I hit this first using Attic 0.14, but then read the "Does attic have
>> an upper data size limit?" thread and since the exception there looks
>> similar I uninstall 0.14 then installed the latest version from git,
>> and I'm still seeing the same issue.
>>
>> I've dug into the code enough to see that the error is on a different
>> line than the fix in the other thread (attic/_hashindex.c line 165 vs
>> line 180).
>>
>> I'm not sure where I should go from here ...
>>
>> Evan Hempel
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1
>>
>> iQIcBAEBAgAGBQJU1YDdAAoJEBQYbQX1fj+hnnMQAIq8nJ4Kn6ZzX+T2A0kSG3Ns
>> L537NjgBYNRdMOyhDkPvI6UVnF+X6NcdBTg9iu6tVMrRNBVql0SdA+7oSKehOMow
>> KYlBFTnt7BnbJxGOYfi3zdu6VqpDMWdnFQwcObfBppFT22ScuRDpD8OibN11PYZ/
>> Us+9TlX3zo85Bl+3OPwTiq47saNDew/8J9XcWuqMwCu7X3D6oAoWIWTxQbQMl1qZ
>> RPxzxJgT3LjkA53gYL0MbGB20MxWKEU95FrX8haK5y6djpO+0w41ihlmECk7bPuh
>> mWQEyo5Rs7OsRK41fmA+9HpFFHuZgMPpdUwn7ELxUzcAC5N/PylOsH5g00zNEQ6b
>> 1dXbZiyD1gKizGclPdVpt4vEgEvR7GpnJ22M9kglYef5u4aAhQicV2xCTG70c7gv
>> ZgVrIOg+heeEi69uy4OUGHhc4PyZertslpXARZbg4dED3XudB4mvU25UdXtkUF9L
>> gODsetA25m9cgYNblbK4T4nBM9ziXlqGZdWn6claQXZ1fmFQRl/SnfRCf5WRnlCA
>> AG40cT3BUZvO7UV0empAGsBwJtDtJKNq40203L/16T5WFxxRIKgwqyvkB/Qe37Tf
>> J6/N5Iio+fJvVwRY/2DA1DGurViKqxvuPXrc7/JfJXgWkxGysIF5wHA3q5+D1FI3
>> 4aZmHh2CmFIoOXn/U6vF
>> =r3fX
>> -----END PGP SIGNATURE-----
>>
>
>
>
> --
> Thiago Coutinho
>
> "O povo não deveria temer o governo. O governo é quem deveria temer o
> povo."
> V de Vingança
>



-- 
Thiago Coutinho

"O povo não deveria temer o governo. O governo é quem deveria temer o povo."
V de Vingança