By default, when you try delete an email in Horde Webmail, it will cross it/put a line through it. That is, it marks it for deletion instead of actually removing it from the account. To delete these emails, you would then need to 'purge deleted emails'.
You can change how Horde handles mail deletion by clicking on the Settings icon (cog icon) > Preferences > Mail. Click 'Deleting and Moving Messages'
Tick 'Move deleted messages to your Trash mailbox instead of marking them as deleted in the current mailbox?' and click Save. Now when you delete an email it will go straight to your Trash folder.
If you require further information/assistance on this, please Open Ticket and a member of the team will be happy to assist further.