diff options
| author | Denys Vlasenko <dvlasenk@redhat.com> | 2016-04-08 17:51:54 +0200 |
|---|---|---|
| committer | David S. Miller <davem@davemloft.net> | 2016-04-13 22:40:54 -0400 |
| commit | f9a7cbbf18f1640907d6ca345b8337e4b50ea56f (patch) | |
| tree | 5ebce0eeac99adc62f6c489abd0dcdbef2520003 /include/net/sock.h | |
| parent | bf91795e4a77eb75602702e4c4d9b98b155039e9 (diff) | |
| download | linux-f9a7cbbf18f1640907d6ca345b8337e4b50ea56f.tar.gz linux-f9a7cbbf18f1640907d6ca345b8337e4b50ea56f.tar.xz | |
net: force inlining of netif_tx_start/stop_queue, sock_hold, __sock_put
Sometimes gcc mysteriously doesn't inline
very small functions we expect to be inlined. See
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66122
Arguably, gcc should do better, but gcc people aren't willing
to invest time into it, asking to use __always_inline instead.
With this .config:
http://busybox.net/~vda/kernel_config_OPTIMIZE_INLINING_and_Os,
the following functions get deinlined many times.
netif_tx_stop_queue: 207 copies, 590 calls:
55 push %rbp
48 89 e5 mov %rsp,%rbp
f0 80 8f e0 01 00 00 01 lock orb $0x1,0x1e0(%rdi)
5d pop %rbp
c3 retq
netif_tx_start_queue: 47 copies, 111 calls
55 push %rbp
48 89 e5 mov %rsp,%rbp
f0 80 a7 e0 01 00 00 fe lock andb $0xfe,0x1e0(%rdi)
5d pop %rbp
c3 retq
sock_hold: 39 copies, 124 calls
55 push %rbp
48 89 e5 mov %rsp,%rbp
f0 ff 87 80 00 00 00 lock incl 0x80(%rdi)
5d pop %rbp
c3 retq
__sock_put: 6 copies, 13 calls
55 push %rbp
48 89 e5 mov %rsp,%rbp
f0 ff 8f 80 00 00 00 lock decl 0x80(%rdi)
5d pop %rbp
c3 retq
This patch fixes this via s/inline/__always_inline/.
Code size decrease after the patch is ~2.5k:
text data bss dec hex filename
56719876 56364551 36196352 149280779 8e5d80b vmlinux_before
56717440 56364551 36196352 149278343 8e5ce87 vmlinux
Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com>
CC: David S. Miller <davem@davemloft.net>
CC: linux-kernel@vger.kernel.org
CC: netdev@vger.kernel.org
CC: netfilter-devel@vger.kernel.org
Signed-off-by: David S. Miller <davem@davemloft.net>
Diffstat (limited to 'include/net/sock.h')
| -rw-r--r-- | include/net/sock.h | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/include/net/sock.h b/include/net/sock.h index baba58770ac5..d997ec13a643 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -569,7 +569,7 @@ static inline bool __sk_del_node_init(struct sock *sk) modifications. */ -static inline void sock_hold(struct sock *sk) +static __always_inline void sock_hold(struct sock *sk) { atomic_inc(&sk->sk_refcnt); } @@ -577,7 +577,7 @@ static inline void sock_hold(struct sock *sk) /* Ungrab socket in the context, which assumes that socket refcnt cannot hit zero, f.e. it is true in context of any socketcall. */ -static inline void __sock_put(struct sock *sk) +static __always_inline void __sock_put(struct sock *sk) { atomic_dec(&sk->sk_refcnt); } |