Within the settings of WooCommerce Print Address Labels, you can easily add custom CSS. As a starting point, you can look at the default stylesheet in the template folder of the plugin.
The main selectors you can use:
- @page (for controlling page properties)
- table (the main table containing the labels for each page)
- td.label (the table cell containing the address block)
- .address-block (a div containing the address data as specified in the plugin settings.)
For example, if you want to change the default vertical alignment of the address, from middle to top, you can enter the following in the Custom styles setting, under WooCommerce > Address Labels > Page layout:
td.label { vertical-align: top; padding-top: 3mm; }
If you want to add HTML (including images/logos), you can do this via the Address/order data settings. This page describes how to do that: Formatting the address
For complete control over the label layout, you can create a custom label template.